Substation anomaly inspection method and device, storage medium and computer equipment

Abstract

The application discloses a substation anomaly inspection method and device, a storage medium and a computer device. The method comprises the following steps: in response to a substation equipment inspection task instruction, a target image of a substation equipment is acquired; according to a shooting point of the target image, a forward sample image is acquired from a forward sample library; according to pixel information of the target image and pixel information of the forward sample image, the target image is marked as a normal target image or a pending target image; when the target image is marked as the pending target image, an image similarity between the target image and the forward sample image is calculated, and whether the target image is an abnormal image is judged according to the image similarity; if the target image is the abnormal image, an abnormal sample image corresponding to the shooting point is acquired from an abnormal sample library, a difference value between the target image and the abnormal sample image is calculated, and when the shooting point is determined as an abnormal inspection point according to the difference value, the target image is reconstructed, and the reconstructed target image is saved into the abnormal sample library.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of substation inspection, and in particular to a substation abnormality inspection method and device, a storage medium and a computer device. BACKGROUND

[0002] As an important part of the power grid, substations have the characteristics of a large number of devices, a complex environment, and a wide geographical distribution. In traditional substation operation and maintenance management, operation and maintenance personnel periodically inspect substation devices manually to master the operation of the substation devices and find device operation abnormalities. In the existing related technology, the number of substation devices increases, and existing device defect sample images cannot cover all device defect types. Substation operation and maintenance personnel only use device defect image recognition results as inspection references and still perform daily inspections by reading images and taking meter readings, which leads to a large workload for operation and maintenance personnel and low accuracy of substation device defect identification. SUMMARY

[0003] Therefore, the present application provides a substation abnormality inspection method and device, a storage medium, and a computer device, which help to improve the accuracy of substation device defect identification.

[0004] According to one aspect of the present application, a substation abnormality inspection method is provided, which includes:

[0005] In response to a substation device inspection task instruction, a target image of the substation device is obtained;

[0006] According to a shooting point of the target image, a forward sample image corresponding to the target image is obtained in a forward sample library;

[0007] According to pixel information of the target image and pixel information of the forward sample image, the target image is marked as a normal target image or a pending target image, and when the target image is marked as a normal target image, the target image marked as a normal target image is subjected to image reconstruction, and the reconstructed target image is saved to the forward sample library;

[0008] When the target image is marked as a pending target image, an image similarity between the target image and the forward sample image is calculated, and whether the target image is an abnormal image is determined according to the image similarity;

[0009] If the target image is an abnormal image, an abnormal sample image corresponding to the shooting point is obtained in an abnormal sample library, a difference value between the target image and the abnormal sample image is calculated, and when the shooting point is determined to be an abnormal inspection point according to the difference value, the target image is subjected to image reconstruction, and the reconstructed target image is saved to the abnormal sample library.

[0010] Optionally, the obtaining the target image of the power transformation station device comprises:

[0011] receiving a pre-detection image photographed by the inspection device at a pre-prepared photographing point, identifying the pre-detection image, obtaining pixel values of each pixel point in the pre-detection image, and calculating quality data of the pre-detection image according to the pixel values;

[0012] if the quality data is greater than or equal to a preset quality threshold, performing denoising on the pre-detection image and determining the pre-detection image after denoising as a target image;

[0013] if the quality data is less than the preset quality threshold, generating a secondary photographing instruction according to the photographing point of the pre-detection image, photographing a new pre-detection image, and obtaining a target image satisfying the preset quality threshold until the target image is obtained.

[0014] Optionally, the marking the target image as a normal target image or a pending target image according to the pixel information of the target image and the pixel information of the forward sample image comprises:

[0015] performing subtraction operation on the pixel values of each pixel point in the target image and the pixel values of the pixel points at the corresponding positions in the forward sample image;

[0016] marking the target image as a normal target image or a pending target image according to the operation result.

[0017] Optionally, after the obtaining the target image of the power transformation station device, the method further comprises:

[0018] calculating a scale space of the target image according to a horizontal coordinate, a vertical coordinate, a standard deviation of a normal distribution, and a preset scale space factor of the target image;

[0019] constructing a Gaussian pyramid based on the scale space and performing Gaussian blur operation to obtain corner point information of the target image;

[0020] projecting the target image onto a forward sample image according to the corner point information, and aligning the target image with the forward sample image.

[0021] Optionally, the image reconstruction on the target image marked as a normal target image and saving the reconstructed target image to the forward sample library comprises:

[0022] identifying high-dimensional feature information of the target image through a convolutional neural network, generating forward feature information based on the high-dimensional feature information, and constructing a forward reconstruction image according to the forward feature information, and saving the forward reconstruction image as a forward sample image of the photographing point in the forward sample library.

[0023] Optionally, the determining whether the target image is an abnormal image according to the image similarity comprises:

[0024] If the image similarity is less than or equal to a preset similarity threshold, the target image is determined as a normal target image, and the target image is saved to the positive sample library.

[0025] If the image similarity is greater than the preset similarity threshold, the target image is determined as an abnormal image.

[0026] Optionally, when the photographing point is determined as an abnormal inspection point according to the difference value, the target image is reconstructed, and the reconstructed target image is saved to an abnormal sample library, comprising:

[0027] When the difference value is less than or equal to a preset difference value, the photographing point is determined as an abnormal inspection point, abnormal inspection information of the photographing point is generated according to the target image, abnormal inspection information alarm is performed, and the target image is reconstructed as an abnormal image, and the reconstructed target image is saved to the abnormal sample library.

[0028] When the image difference value is greater than the preset difference value, the target image is sent to an operation and maintenance terminal, so that an operation and maintenance personnel performs abnormal review on the target image, and the target image is reconstructed according to the received abnormal review result, and the reconstructed target image is saved to the positive sample library or the abnormal sample library, wherein the image reconstruction comprises positive image reconstruction and abnormal image reconstruction.

[0029] According to another aspect of the present application, a substation abnormal inspection device is provided, the device comprising:

[0030] A first acquisition module is configured to acquire a target image of a substation device in response to a substation device inspection task instruction.

[0031] A second acquisition module is configured to acquire a positive sample image corresponding to the target image from a positive sample library according to a photographing point of the target image.

[0032] A marking module is configured to mark the target image as a normal target image or a pending target image according to pixel information of the target image and pixel information of the positive sample image, and to reconstruct the target image marked as the normal target image when the target image is marked as the normal target image, and save the reconstructed target image to the positive sample library.

[0033] a judging module, configured to calculate an image similarity between the target image and the positive sample image when the target image is marked as a pending target image, and determine whether the target image is an abnormal image according to the image similarity;

[0034] a determining module, configured to, if the target image is an abnormal image, acquire an abnormal sample image corresponding to the shooting point from an abnormal sample library, calculate a difference value between the target image and the abnormal sample image, and determine the shooting point as an abnormal inspection point according to the difference value, and perform image reconstruction on the target image, and save the reconstructed target image to the abnormal sample library.

[0035] Optionally, the first acquiring module is further configured to:

[0036] receive a pre-detection image shot by an inspection device at a pre-prepared shooting point, identify the pre-detection image, acquire pixel values of each pixel point in the pre-detection image, and calculate quality data of the pre-detection image according to the pixel values;

[0037] if the quality data is greater than or equal to a preset quality threshold, perform denoising on the pre-detection image and determine the denoised pre-detection image as a target image;

[0038] if the quality data is less than the preset quality threshold, generate a secondary shooting instruction according to the shooting point of the pre-detection image, shoot a new pre-detection image, and repeat the process until a target image satisfying the preset quality threshold is obtained.

[0039] Optionally, the marking module is further configured to:

[0040] perform subtraction operation on the pixel values of each pixel point in the target image and the pixel values of pixel points at corresponding positions in the positive sample image;

[0041] according to the operation result, mark the target image as a normal target image or a pending target image.

[0042] Optionally, the apparatus further comprises an aligning module configured to:

[0043] calculate a scale space of the target image according to a horizontal coordinate, a vertical coordinate of the target image, a standard deviation of a normal distribution, and a preset scale space factor;

[0044] construct a Gaussian pyramid based on the scale space and perform Gaussian blur operation to acquire corner point information of the target image;

[0045] project the target image onto the positive sample image according to the corner point information, and align the target image with the positive sample image.

[0046] Optionally, the apparatus further comprises a reconstruction module configured to:

[0047] The high-dimensional feature information of the target image is identified by the convolutional neural network, the forward feature information is generated based on the high-dimensional feature information, and the forward reconstruction image is constructed according to the forward feature information, and the forward reconstruction image is saved in the forward sample library as the forward sample image of the shooting point.

[0048] Optionally, the judging module is further configured to:

[0049] If the image similarity is less than or equal to the preset similarity threshold, the target image is determined as a normal target image, and the target image is saved to the forward sample library.

[0050] If the image similarity is greater than the preset similarity threshold, the target image is determined as an abnormal image.

[0051] Optionally, the determining module is further configured to:

[0052] When the difference value is less than or equal to the preset difference value, the shooting point is determined as an abnormal inspection point, the abnormal inspection information of the shooting point is generated according to the target image, the abnormal inspection information alarm is performed, the abnormal image reconstruction of the target image is performed, and the reconstructed target image is saved to the abnormal sample library.

[0053] When the image difference value is greater than the preset difference value, the target image is sent to an operation and maintenance terminal, so that an operation and maintenance personnel performs abnormal review on the target image, and the target image is reconstructed according to the received abnormal review result, and the reconstructed target image is saved to the forward sample library or the abnormal sample library, wherein the image reconstruction includes forward image reconstruction and abnormal image reconstruction.

[0054] According to another aspect of the present application, a storage medium having a computer program stored thereon is provided, and the program is executed by a processor to implement the substation abnormal inspection method.

[0055] According to another aspect of the present application, a computer device is provided, which comprises a storage medium, a processor, and a computer program stored in the storage medium and executable on the processor, and the processor implements the substation abnormal inspection method when executing the program.

[0056] By the technical scheme, the substation abnormal inspection method and device, the storage medium and the computer equipment provided by the application are used to respond to a substation equipment inspection task instruction, obtain a target image of a substation equipment, obtain a forward sample image corresponding to the target image in a forward sample library according to a shooting point of the target image, mark the target image as a normal target image or a pending target image according to pixel information of the target image and pixel information of the forward sample image, perform image reconstruction on the target image marked as the normal target image when the target image is marked as the normal target image, save the reconstructed target image to the forward sample library, calculate an image similarity between the target image and the forward sample image when the target image is marked as the pending target image, determine whether the target image is an abnormal image according to the image similarity, obtain an abnormal sample image corresponding to the shooting point in an abnormal sample library if the target image is the abnormal image, calculate a difference value between the target image and the abnormal sample image, and perform image reconstruction on the target image when it is determined that the shooting point is an abnormal inspection point according to the difference value, and save the reconstructed target image to the abnormal sample library. The current substation equipment inspection image is compared with the forward sample library to filter out the normal target image to enrich the forward sample library, the remaining pending target image is subjected to abnormal identification again to filter out the abnormal image to enrich the abnormal sample library, and the normal target image enriches the forward sample library, thereby improving the identification accuracy of the abnormal image and the substation abnormal inspection efficiency.

[0057] The above description is only a summary of the technical scheme of the application. In order to more clearly understand the technical means of the application, the application can be implemented according to the content of the description, and in order to make the above and other purposes, characteristics and advantages of the application more obvious and easy to understand, the following specific embodiments of the application are described. BRIEF DESCRIPTION OF DRAWINGS

[0058] The drawings described herein are used to provide further understanding of the application, and form a part of the application. The schematic embodiments of the application and the description thereof are used to explain the application, and do not constitute an improper limitation on the application. In the drawings:

[0059] Figure 1 A flowchart of a substation abnormal inspection method provided by an embodiment of the application is shown;

[0060] Figure 2 A flowchart of another substation abnormal inspection method provided by an embodiment of the application is shown;

[0061] Figure 3 A structure diagram of a substation abnormal inspection device provided by an embodiment of the application is shown;

[0062] Figure 4A schematic diagram of shooting point information of substation image inspection prefabrication provided by an embodiment of the present application is shown.

[0063] Figure 5 A flowchart of a similarity comparison model of substation equipment image and positive sample library image provided by an embodiment of the present application is shown.

[0064] Figure 6 A flowchart of another substation abnormality inspection method provided by an embodiment of the present application is shown. DETAILED DESCRIPTION

[0065] The present application will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments. It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0066] In the present embodiment, a substation abnormality inspection method is provided, as shown in the figure, the method comprises: Figure 1

[0067] Step 101, in response to a substation equipment inspection task instruction, obtaining a target image of the substation equipment.

[0068] The above embodiments of the present application can be applied to the inspection of a substation, for identifying abnormalities in substation equipment. First, in response to a substation equipment inspection task instruction, which is issued by a substation maintenance personnel, then in response to the substation equipment inspection task instruction, controlling an inspection device in the substation to perform video shooting inspection on the substation equipment, obtaining a target image of the operation of the equipment in the substation, the inspection device including a high-definition video camera, a network video recorder, etc., in order to prepare for later completion of abnormality inspection.

[0069] Step 102, according to the shooting point of the target image, obtaining a positive sample image corresponding to the target image in a positive sample library.

[0070] Next, according to the shooting point of the target image, obtaining a positive sample image corresponding to the target image in a positive sample library, wherein the target image at each shooting point is obtained, and the positive sample images of all shooting points are stored in the positive sample library, for example, the shooting points can be set as needed, specifically, obtaining a positive sample image matching the shooting points in the positive sample library, in order to prepare for later completion of abnormality inspection.

[0071] Step 103, according to the pixel information of the target image and the pixel information of the positive sample image, marking the target image as a normal target image or a pending target image

[0072] ​Next, the pixel information of the target image and the pixel information of the forward sample image are obtained, and whether the target image is a normal target image or a pending target image is marked according to the two pixel information, that is, whether the target image exists abnormity is marked according to the two pixel information, wherein the normal target image is a normal target image without abnormity, and the pending target image is a target image which is uncertain to be a normal target image and needs to be further determined, and the normal target image is filtered out through the pixel information of the image, so as to prepare for the next abnormal inspection.

[0073] Step 104, when the target image is marked as a normal target image, image reconstruction is performed on the target image marked as a normal target image, and the reconstructed target image is saved to the forward sample library; when the target image is marked as a pending target image, the image similarity between the target image and the forward sample image is calculated, and whether the target image is an abnormal image is determined according to the image similarity.

[0074] Next, when the target image is marked as a normal target image, image reconstruction is performed on the target image, and the reconstructed target image is saved to the forward sample library as a forward sample image, thereby increasing the number of forward sample images in the forward sample library; when the target image is marked as a pending target image, the image similarity between the target image and the forward sample image is calculated, and whether the target image is an abnormal image is determined according to the image similarity, thereby judging again whether the pending target image exists abnormity through the calculation of the image similarity, improving the accuracy of identifying the abnormality of the target image, and further preparing for the next abnormal inspection.

[0075] Step 105, if the target image is an abnormal image, an abnormal sample image corresponding to the shooting point is obtained from the abnormal sample library, the difference value between the target image and the abnormal sample image is calculated, and when it is determined that the shooting point is an abnormal inspection point according to the difference value, image reconstruction is performed on the target image, and the reconstructed target image is saved to the abnormal sample library.

[0076] Next, if the pending target image is an abnormal image, an abnormal sample image corresponding to the shooting point is obtained from the abnormal sample library, the difference value between the target image and the abnormal sample image is calculated, and whether the shooting point is an abnormal inspection point is determined according to the difference value, and when it is determined that the shooting point is an abnormal inspection point, image reconstruction is performed on the target image, and the reconstructed target image is saved to the abnormal sample library, thereby completing the substation equipment inspection task, and the sample number of the forward sample image and the abnormal sample image is increased in the whole inspection process, thereby improving the accuracy of the abnormal inspection.

[0077] By applying the technical solution of the embodiment, in response to a substation equipment inspection task instruction, a target image of the substation equipment is acquired; according to a shooting point of the target image, a forward sample image corresponding to the target image is acquired in a forward sample library; according to pixel information of the target image and pixel information of the forward sample image, the target image is marked as a normal target image or a pending target image, and when the target image is marked as the normal target image, the target image marked as the normal target image is subjected to image reconstruction, and the reconstructed target image is saved to the forward sample library; when the target image is marked as the pending target image, an image similarity between the target image and the forward sample image is calculated, and whether the target image is an abnormal image is determined according to the image similarity; if the target image is the abnormal image, an abnormal sample image corresponding to the shooting point is acquired in an abnormal sample library, a difference value between the target image and the abnormal sample image is calculated, and when the shooting point is determined to be an abnormal inspection point according to the difference value, the target image is subjected to image reconstruction, and the reconstructed target image is saved to the abnormal sample library. By comparing the current substation equipment inspection image with the forward sample library, the normal target image is filtered out to enrich the forward sample library, the remaining pending target image is subjected to abnormal identification again, the abnormal image is filtered out to enrich the abnormal sample library, and the normal target image enriches the forward sample library, thereby improving the identification precision of the abnormal image and further improving the substation abnormal inspection efficiency.

[0078] Further, as a refinement and expansion of the above embodiment, in order to completely describe the specific implementation process of the embodiment, another substation abnormal inspection method is provided, as shown in Figure 2 The method comprises the following steps.

[0079] In step 201, in response to a substation equipment inspection task instruction, a pre-detection image shot by an inspection device at a pre-prepared shooting point is received, the pre-detection image is identified, pixel values of each pixel point in the pre-detection image are acquired, and quality data of the pre-detection image is calculated according to the pixel values.

[0080] In the above embodiment of the application, first, in response to a substation equipment inspection task instruction, then, a pre-detection image of each pre-prepared shooting point is shot by an inspection device at each pre-prepared shooting point, for example, as shown in Figure 3 The pixel values of each pixel point in each pre-detection image are acquired, and the quality data of each pre-detection image is calculated according to the pixel values, for example, the quality data comprises gradient information and image peak signal-to-noise ratio, and the gradient of the pre-detection image is calculated according to the following formula

[0081]

[0082] s(EX ,E y ) is the input pre-detection image; is the sum of the second-order derivatives of the pre-detection image in the horizontal coordinate direction, is the sum of the second-order derivatives of the pre-detection image in the vertical coordinate direction, and then the root mean square difference R of the pre-detection image is calculated according to the following formula mse :

[0083]

[0084] wherein n a is the horizontal coordinate pixel value of the pre-detection image, o a is the vertical coordinate pixel value of the pre-detection image, H ij is the pre-detection image after adding noise information, E ij is the original pre-detection image, and then the image peak signal-to-noise ratio L of the pre-detection image is calculated according to the following formula ps :

[0085]

[0086] wherein A max is the maximum pixel value of the pre-detection image, and is prepared for the next abnormal inspection.

[0087] Step 202, if the quality data is greater than or equal to the preset quality threshold, the pre-detection image is denoised and the pre-detection image after denoising is determined as the target image; if the quality data is less than the preset quality threshold, a secondary shooting instruction is generated according to the shooting point of the pre-detection image, a new pre-detection image is shot, and the target image satisfying the preset quality threshold is obtained until the target image satisfying the preset quality threshold is obtained.

[0088] Next, the quality data and the preset quality threshold are compared, and first, the preset quality threshold Δf1 is calculated according to the following formula:

[0089]

[0090] wherein n p is the number of images in the positive sample library, f a is the current gradient information, f bi is the gradient value of different positive sample library images, f bmax -f bmin is the image gradient offset value, f bmax is the maximum image gradient in the positive sample library, f bmin is the minimum image gradient in the positive sample library; if the gradient or the image peak signal-to-noise ratio L psgreater than or equal to the preset quality threshold Af1, the pre-detection image is denoised, and the pre-detection image after denoising is determined as the target image; if the gradient and the image peak signal-to-noise ratio L ps are both less than the preset quality threshold Af1, a secondary shooting instruction is generated according to the shooting point of the pre-detection image, a new pre-detection image is shot, and the target image satisfying the preset quality threshold is obtained until the image quality of the target image is detected, the definition of the target image is improved, and preparation is made for the subsequent abnormal inspection.

[0091] It should be noted that, due to environmental factors, there is noise information in the process of shooting the substation equipment image. Although the quality analysis and retake of the pre-detection image can reduce the noise data in the pre-detection image, the noise information cannot be completely eliminated, and therefore, the pre-detection image needs to be denoised. In this embodiment, the pre-detection image is denoised by opening operation; the denoising process is as follows:

[0092] The pre-detection image is eroded as E f :

[0093]

[0094] wherein, l z is a structure element of the pre-detection image, q c is a type of erosion template, and E a is the pre-detection image subjected to denoising processing of the input image.

[0095] The pre-detection image is dilated as E g :

[0096] E g ={q c | l z ∩E f}

[0097] wherein, l z is a structure element of the pre-detection image, q c is a type of erosion template, and E f is an eroded image of the pre-detection image.

[0098] The method first erodes the pre-detection image and then dilates it, so as to reduce the noise information such as burrs in the pre-detection image. The erosion of the pre-detection image compresses the highlight area in the image and reduces the noise data such as burrs, so as to reflect the actual characteristics of the pre-detection image. The dilation of the pre-detection image restores the actual value of the highlight area in the pre-detection image. Through the erosion and dilation processing of the pre-detection image, the noise information in the pre-detection image is reduced, and the accuracy of identifying the target image is improved.

[0099] Step 203: Calculate the scale space of the target image based on its horizontal coordinates, vertical coordinates, standard deviation of normal distribution, and preset scale space factor; construct a Gaussian pyramid based on the scale space and perform Gaussian blur operation to obtain the corner information of the target image; project the target image onto the forward sample image based on the corner information, and align the target image with the forward sample image.

[0100] Next, based on the horizontal and vertical coordinates, the standard deviation of the normal distribution, and the preset scale space factor of the target image, the scale space of the target image is calculated. For example, the Gaussian function G is first determined according to the following formula. xy :

[0101]

[0102] Where, x c Let y be the horizontal coordinate of the target image. c Let be the vertical coordinate of the target image, δ be the standard deviation of the normal distribution of the target image, and e be the vertical coordinate of the target image. b Let k be the scale space factor. Calculate the scale space k of the target image above using the following formula. xy :

[0103] k xy =G xy *E xy

[0104] Among them, E xy The input is the target image, and * represents the convolution operator;

[0105] Then, a Gaussian pyramid is constructed based on the aforementioned scale space, and a Gaussian blur operation is performed to obtain the corner information of the target image. Based on this corner information, the target image is projected onto the forward sample image, and the target image and the forward sample image are aligned. For example, projection is performed according to the following formula:

[0106]

[0107] Where, x d y d z d Let x be the x, y, and y coordinates of the target image after perspective projection. b y b z b To determine the x, y, and y coordinates of the target image before perspective projection, b 11 b 12 b 13 b 21 b 22 b23 , b 31 , b 32 , b 33 , is a perspective transformation matrix of 3*3, by aligning the target image and the forward sample image, further improving the accuracy of the subsequent abnormal inspection.

[0108] Step 204, according to the shooting point of the target image, the corresponding forward sample image is obtained in the forward sample library; the pixel value of each pixel point in the target image is subtracted from the pixel value of the corresponding pixel point in the forward sample image, and according to the operation result, the target image is marked as a normal target image or a pending target image.

[0109] Next, according to the shooting point of the target image, for example, the radiator of the main transformer, the corresponding forward sample image of the radiator of the main transformer is obtained in the forward sample library; the pixel value of each pixel point in the target image is subtracted from the pixel value of the corresponding pixel point in the forward sample image, and a plurality of calculation difference values are obtained, if more than 80% of the calculation difference values are within the preset difference value range, the target image is marked as a normal target image, if not, the target image is marked as a pending target image. The target image is identified by pixel value, and the target image is determined as a normal target image or a pending target image, wherein the normal target image is used for forward sample training library after forward image reconstruction, which further improves the accuracy of abnormal inspection.

[0110] Step 205, when the target image is marked as a normal target image, the high-dimensional feature information of the target image is identified by convolutional neural network, the forward feature information is generated based on the high-dimensional feature information, and the forward reconstruction image is constructed according to the forward feature information. The forward reconstruction image is saved as the forward sample image of the shooting point in the forward sample library.

[0111] Next, when the target image is marked as a normal target image, the high-dimensional feature information of the target image is identified by convolutional neural network (CNN), the forward feature information is generated based on the high-dimensional feature information, and the forward reconstruction image is constructed according to the forward feature information. The forward reconstruction image is saved as the forward sample image of the shooting point in the forward sample library, for example, after the normal target image is input into the convolutional neural network:

[0112] The process of CNN convolution is to calculate the sum of the weight and brightness of the normal target image, and the CNN convolution is C xy :

[0113]

[0114] Where: ne is the size of the convolution kernel of the CNN, o ei is the brightness of different normal target images, w ei is the weight of different normal target images, after the convolution of the normal target images, the bias is added and activated to obtain the result C ef :

[0115]

[0116] wherein β is the bias of the CNN, γ is the sigmoid activation function in the CNN, then the pooling is performed to reduce the feature space of the substation equipment image; finally, the full connection is performed, and the forward reconstruction image output is C g :

[0117]

[0118] wherein n g is the number of layers of neurons of the CNN, w gi is the weight of different CNN neuron layers, u i-1 is the output of the previous layer of neurons of the CNN different layers; the above forward reconstruction image C g is saved as the forward sample image of the above shooting point. Prepare for the next abnormal inspection.

[0119] Step 206, when the target image is marked as a pending target image, the image similarity between the target image and the forward sample image is calculated, if the image similarity is less than or equal to a preset similarity threshold, the target image is determined as a normal target image, and the target image is saved to the forward sample library.

[0120] Next, when the above target image is marked as a pending target image, the image similarity between the above target image and the above forward sample image is calculated, for example, as shown in Figure 4 , C0 is the forward sample image feature vector; C1 is the current input target image feature vector, first, C0 and C1 are subjected to 11x11 step 4 convolution and ReLU activation function to obtain C2; secondly, C0 and C1 are subjected to maximum pooling, 5x5 step 2 convolution and ReLU activation function to obtain C3; then, C0 and C1 are subjected to maximum pooling, 3x3 step 1 convolution and Sigmoid activation function to obtain C4; finally, C2, C3 and C4 are weighted and averaged to obtain the forward sample similarity score, and the similarity of the above target image and the forward sample image is d ab :

[0121]

[0122] wherein nb n is the number of layers in learning perceptual image patch similarity (LPIPS) ; c w is the number of combinations of scores and weights in LPIPS; i v is the trainable weight of different channels and dimensions; i w is the mapping score of different channels; l y is the sum of LPIPS channel and dimension weights. aj and y bj are the values of the current target image and the forward sample image after feature extraction and unit normalization, respectively; wherein the above-mentioned preset similarity threshold Δf2 is calculated according to the following formula:

[0123]

[0124] wherein f c is the current similarity score, f cavg is the average value of the similarity between the forward sample library images, f cmax -f cmin is the image similarity offset value, f cmax is the maximum image similarity in the forward sample library; f cmin is the minimum image similarity in the forward sample library;

[0125] If the above-mentioned image similarity d ab is less than or equal to the preset similarity threshold Δf2, it is determined that the above-mentioned target image is a normal target image, and the above-mentioned target image is saved to the above-mentioned forward sample library. For the pending target image, the image similarity is calculated again to identify whether the above-mentioned target and image are abnormal images, and the pending target image is identified again through the image similarity, which improves the accuracy of identifying the target image and prepares for the next abnormal inspection.

[0126] Step 207, if the image similarity is greater than the preset similarity threshold, it is determined that the target image is an abnormal image, then the abnormal sample image corresponding to the shooting point is obtained in the abnormal sample library, the difference value between the target image and the abnormal sample image is calculated, and when the difference value is less than or equal to the preset difference value, it is determined that the shooting point is an abnormal inspection point, the abnormal inspection information of the shooting point is generated according to the target image, the abnormal inspection information alarm is performed, and the abnormal image reconstruction of the target image is performed, and the reconstructed target image is saved to the abnormal sample library.

[0127] Next, if the above-mentioned image similarity d abgreater than a preset similarity threshold Af2, the target image is determined as an abnormal image, an abnormal sample image corresponding to the shooting point is obtained from an abnormal sample library, and a difference value between the target image and the abnormal sample image is calculated. For example, first, an abnormal sample average value z is obtained based on the abnormal feature value and a preset third-layer target image defect channel number n1 by using the following formula a3 :

[0128]

[0129] wherein n1 is the third-layer target image defect channel number; d max3i is a different third-layer channel target image defect maximum difference value;

[0130] Then, the abnormal sample average value z a3 is sampled on a third saliency map layer and added to a second-layer average image defect sample value to obtain a defect difference value z a2 :

[0131]

[0132] wherein n0 is the second-layer target image defect channel number; d max2i is a different second-layer channel target image defect maximum difference value; z ah3 is a saliency map sampling value of z a3 ;

[0133] Finally, the second defect difference value z a2 is sampled on a second saliency map layer and added to a third-layer average image defect value to obtain a first-layer output target image defect value, i.e., a difference value z a1 between the target image and the abnormal sample image:

[0134]

[0135] wherein n p is the first-layer target image defect channel number; d max1i is a different first-layer channel target image defect maximum difference value; z ah2 is a saliency map sampling value of z a2 ;

[0136] When the difference value z a1When the image difference value is less than or equal to the preset difference value, the shooting point is determined as an abnormal inspection point, abnormal inspection information of the shooting point is generated according to the target image, the abnormal inspection information is generated according to the abnormal inspection information, an abnormal inspection alarm is given, the target image is reconstructed as an abnormal image, the reconstructed target image is saved into an abnormal sample library, the abnormal sample library is updated again by calculating the difference value between the target image and the abnormal image sample, the number of abnormal sample images in the abnormal sample library is increased, and the accuracy of the abnormal inspection is improved.

[0137] In step 208, when the image difference value is greater than the preset difference value, the target image is sent to an operation and maintenance terminal, so that an operation and maintenance personnel performs abnormal review on the target image, and the target image is reconstructed according to the received abnormal review result, and the reconstructed target image is saved into a positive sample library or an abnormal sample library, wherein the image reconstruction includes positive image reconstruction and abnormal image reconstruction.

[0138] Next, when the image difference value z a1 When the image difference value is greater than the preset difference value, the target image is sent to an operation and maintenance terminal, so that an operation and maintenance personnel performs abnormal review on the target image, for example, semi-supervised learning review (SSL), when it is received that the target image belongs to an abnormal image, defect information is labeled, the target image is reconstructed as an abnormal image, the reconstructed target image is saved into an abnormal sample library, is used for training of the abnormal sample library, and an abnormal information alarm is given; when it is received that the target image belongs to a normal image, the target image is reconstructed as a positive image, the reconstructed target image is saved into a positive sample library, is used for training of the positive sample library, so that the accuracy of the abnormal inspection is improved, and the efficiency of the abnormal inspection is further improved.

[0139] It should be noted that when the positive sample library is established, the high-dimensional feature memory is used to remember the features of the positive sample image, so that the high-dimensional features of the image after model training are consistent with the memory value; in the model memory process, the positive sample image is input into the model, the features of the positive sample image are extracted in each block of the CNN, and memory information is formed; in the inference stage, the CNN high-dimensional feature code takes the high-dimensional features from the positive sample library, and compares them with the current memory information to form the difference between the input positive sample image and the positive sample library:

[0140]

[0141] Wherein, n h is the number of positive samples in the positive sample library, M bi is different input positive sample images; M aiThe positive sample image in the different positive sample library; take the image with the largest difference from the current sample as the maximum allowed range of positive sample deviation.

[0142] In high-dimensional feature decoding, first, the multiple encoding features of the target image in the CNN memory module are spliced in the CNN channel dimension; second, the spliced target image result is adjusted through convolution; on this basis, the target image result is obtained through the rectified linear unit (ReLU) activation function to obtain the first layer output of the high-dimensional feature decoder; then, the second layer output is obtained through scale adjustment, and all features of the high-dimensional feature decoding are output.

[0143] In semi-supervised learning SSL, the positive sample image labeled by the substation operation and maintenance personnel is used to train the model to recognize unlabeled image samples, thereby making up for the problem of large workload of positive sample labeling. The model training takes the minimum convergence of the mean squared error (MSE) as the target, and with the increase of the number of training times, the MES converges, that is, the model training is completed. The MSE of the positive sample image is:

[0144]

[0145] Where, n k is the number of positive sample images, E ki is the positive value of different positive sample images, H ki is the value of different positive reconstruction images.

[0146] In model training, the activation function ReLU linear rectifier function is used for updating to improve the accuracy of model training. In substation abnormal inspection, if a small image anomaly detection such as high-voltage sleeve oil leakage is performed, the region of interest in the picture is extracted and processed through the semantic segmentation network, the invalid background information is shielded, then the positive image reconstruction is performed, and the substation equipment image positive sample library is formed.

[0147] By applying the technical solution of the embodiment, the target image is compared with the positive sample library, the normal target image is filtered out to enrich the positive sample library, the to-be-determined target image is identified again, the normal target image is filtered out to enrich the positive sample library, and the abnormal image is filtered out to enrich the abnormal sample library. The difference value of the small amount of filtered abnormal images is calculated and reported to the substation operation and maintenance personnel for semi-supervised learning review. The enrichment of the positive sample library and the abnormal sample library improves the accuracy of abnormal inspection, thereby greatly reducing the workload of the substation operation and maintenance personnel, reducing the time consumption of single substation equipment inspection, and further improving the efficiency of substation video inspection.

[0148] Further, the embodiment of the present application provides a substation abnormal inspection method, as shown in the method comprises the following steps. Figure 5

[0149] Firstly, according to the substation inspection task instruction, the substation equipment image is collected, the image quality of the collected substation equipment image is analyzed, for the substation image which does not meet the quality requirement, according to the corresponding shooting point information, the equipment image is re-shot until the equipment image meets the quality requirement, and the equipment image meeting the quality requirement is determined as a target image;

[0150] The target image meeting the quality requirement is subjected to denoising processing and alignment with the image of the corresponding shooting point in the forward sample library;

[0151] After the target image is aligned, the target image is recognized and marked as a normal target image, the normal target image is subjected to forward image reconstruction, used for forward sample training, and saved to the forward sample library;

[0152] For the remaining target images which cannot be marked as normal target images, i.e. pending target images, it is identified whether there is an abnormal area, if there is no abnormal area, the target image is subjected to forward image reconstruction and saved to the forward sample library;

[0153] If there is an abnormal area, the abnormal area information is obtained and compared with the images in the abnormal sample library to determine whether it meets the abnormal characteristics, if it is determined that it meets the abnormal characteristics, the abnormal information is reported and the abnormal sample library is updated;

[0154] If it is not determined whether it meets the abnormal characteristics, the target image is sent to the operation and maintenance personnel for semi-supervised review, if the target image received is a normal target image, the target image is subjected to forward image reconstruction and saved to the forward sample library; if the target image received is an abnormal image, the target image is subjected to abnormal image reconstruction and saved to the abnormal sample library.

[0155] Further, as a specific implementation of the method, Figure 1 the embodiment of the present application provides a substation abnormal inspection device, as shown in the device comprises the following steps. Figure 6

[0156] The first acquisition module is configured to acquire the target image of the substation equipment in response to the substation equipment inspection task instruction;

[0157] The second acquisition module is configured to acquire the forward sample image corresponding to the target image in the forward sample library according to the shooting point of the target image;

[0158] ​​a marking module, configured to mark the target image as a normal target image or a pending target image according to pixel information of the target image and pixel information of the forward sample image, and perform image reconstruction on the target image marked as the normal target image, and save the reconstructed target image to the forward sample library;

[0159] a judging module, configured to calculate an image similarity between the target image and the forward sample image when the target image is marked as the pending target image, and determine whether the target image is an abnormal image according to the image similarity;

[0160] a determining module, configured to, if the target image is the abnormal image, acquire an abnormal sample image corresponding to the shooting point from an abnormal sample library, calculate a difference value between the target image and the abnormal sample image, and perform image reconstruction on the target image when it is determined that the shooting point is an abnormal inspection point according to the difference value, and save the reconstructed target image to the abnormal sample library.

[0161] Optionally, the first acquiring module is further configured to:

[0162] receive a pre-detection image shot by the inspection device at a pre-prepared shooting point, identify the pre-detection image, acquire pixel values of each pixel point in the pre-detection image, and calculate quality data of the pre-detection image according to the pixel values;

[0163] if the quality data is greater than or equal to a preset quality threshold, perform denoising on the pre-detection image and determine the denoised pre-detection image as a target image;

[0164] if the quality data is less than the preset quality threshold, generate a secondary shooting instruction according to the shooting point of the pre-detection image, shoot a new pre-detection image, and repeat the process until a target image meeting the preset quality threshold is obtained.

[0165] Optionally, the marking module is further configured to:

[0166] perform subtraction operation on the pixel values of each pixel point in the target image and pixel values of pixel points at corresponding positions in the forward sample image;

[0167] mark the target image as a normal target image or a pending target image according to the operation result.

[0168] Optionally, the apparatus further includes an aligning module configured to:

[0169] calculate a scale space of the target image according to a horizontal coordinate, a vertical coordinate, a standard deviation of a normal distribution, and a preset scale space factor of the target image;

[0170] construct a Gaussian pyramid based on the scale space, and perform Gaussian blur operation to obtain corner point information of the target image;

[0171] project the target image onto a forward sample image according to the corner point information, and align the target image with the forward sample image.

[0172] Optionally, the apparatus further comprises a reconstruction module configured to:

[0173] identify high-dimensional feature information of the target image through a convolutional neural network, generate forward feature information based on the high-dimensional feature information, and construct a forward reconstruction image according to the forward feature information, and save the forward reconstruction image as the forward sample image of the shooting point in the forward sample library.

[0174] Optionally, the judging module is further configured to:

[0175] if the image similarity is less than or equal to a preset similarity threshold, determine that the target image is a normal target image, and save the target image to the forward sample library;

[0176] if the image similarity is greater than the preset similarity threshold, determine that the target image is an abnormal image.

[0177] Optionally, the determining module is further configured to:

[0178] when the difference value is less than or equal to a preset difference value, determine that the shooting point is an abnormal inspection point, generate abnormal inspection information of the shooting point according to the target image, perform abnormal inspection information alarm, and perform abnormal image reconstruction on the target image, and save the reconstructed target image to an abnormal sample library;

[0179] when the image difference value is greater than the preset difference value, send the target image to an operation and maintenance terminal, so that an operation and maintenance personnel performs abnormal review on the target image, and perform image reconstruction on the target image according to the received abnormal review result, and save the reconstructed target image to the forward sample library or the abnormal sample library, wherein the image reconstruction comprises forward image reconstruction and abnormal image reconstruction.

[0180] It should be noted that other corresponding descriptions of each functional unit involved in the substation abnormal inspection device provided in the embodiments of the present application can be referred to the corresponding descriptions in the method, which will not be repeated here. Figures 1 to 5

[0181] Based on the above method as shown in Figures 1 to 5 correspondingly, the embodiments of the present application further provide a storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the above method as shown in​Figures 1 to 5 The substation anomaly inspection method shown.

[0182] Based on such understanding, the technical scheme of the present application can be embodied in the form of a software product, which can be stored in a nonvolatile storage medium (which can be a CD-ROM, a U disk, a mobile hard disk, etc.), and includes a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the method described in various implementation scenarios of the present application.

[0183] Based on the above method as Figures 1 to 5 The method shown, and Figure 6 In order to achieve the above purpose, the embodiments of the present application also provide a computer device, which can be a personal computer, a server, a network device, etc., and the computer device includes a storage medium and a processor; the storage medium is used to store a computer program; the processor is used to execute the computer program to realize the substation anomaly inspection method as Figures 1 to 5 The method shown.

[0184] Optionally, the computer device can also include a user interface, a network interface, a camera, a radio frequency (RF) circuit, a sensor, an audio circuit, a WI-FI module, etc. The user interface can include a display, an input unit such as a keyboard, etc. The optional user interface can also include a USB interface, a card reader interface, etc. The network interface can optionally include a standard wired interface, a wireless interface (such as a Bluetooth interface, a WI-FI interface), etc.

[0185] Those skilled in the art can understand that the computer device structure provided by the embodiments does not constitute a limitation on the computer device, and can include more or fewer components, or combine certain components, or different component arrangements.

[0186] The storage medium can also include an operating system and a network communication module. The operating system is a program that manages and saves computer device hardware and software resources, and supports the running of information processing programs and other software and / or programs. The network communication module is used to realize the communication between the components inside the storage medium, and the communication with other hardware and software in the entity device.

[0187] Through the description of the above embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software and necessary general hardware platforms, or by hardware, in response to a substation equipment inspection task instruction, a target image of the substation equipment is acquired; according to a shooting point of the target image, a forward sample image corresponding to the target image is acquired in a forward sample library; according to pixel information of the target image and pixel information of the forward sample image, the target image is marked as a normal target image or a pending target image, and when the target image is marked as a normal target image, the target image marked as a normal target image is reconstructed, and the reconstructed target image is saved to the forward sample library; when the target image is marked as a pending target image, an image similarity between the target image and the forward sample image is calculated, and whether the target image is an abnormal image is determined according to the image similarity; if it is an abnormal image, an abnormal sample image corresponding to the shooting point is acquired in an abnormal sample library, a difference value between the target image and the abnormal sample image is calculated, and when it is determined that the shooting point is an abnormal inspection point according to the difference value, the target image is reconstructed, and the reconstructed target image is saved to the abnormal sample library. By comparing the current substation equipment inspection image with the forward sample library, normal target images are filtered out to enrich the forward sample library, the remaining pending target images are subjected to abnormal identification again, abnormal images are filtered out to enrich the abnormal sample library, and the normal target images enrich the forward sample library, thereby improving the identification accuracy of abnormal images and further improving the substation abnormal inspection efficiency.

[0188] Those skilled in the art can understand that the drawings are only schematic diagrams of preferred implementation scenarios, and the modules or flows in the drawings are not necessarily required for implementing the present application. Those skilled in the art can understand that the modules in the devices in the implementation scenarios can be distributed in the devices in the implementation scenarios according to the description of the implementation scenarios, or can be changed and located in one or more devices different from the implementation scenarios. The modules of the above implementation scenarios can be combined into one module, or can be further split into multiple sub-modules.

[0189] The above application numbers are only for description, and do not represent the advantages and disadvantages of the implementation scenarios. The above disclosure is only a few specific implementation scenarios of the present application, but the present application is not limited thereto, and any changes that can be thought of by those skilled in the art should fall within the protection scope of the present application.

Claims

1. A substation anomaly inspection method, characterized by, The method comprises: in response to a substation equipment inspection task instruction, obtaining a target image of the substation equipment; according to a shooting point of the target image, obtaining a forward sample image corresponding to the target image in a forward sample library; according to pixel information of the target image and pixel information of the forward sample image, marking the target image as a normal target image or a pending target image, and when the target image is marked as a normal target image, performing image reconstruction on the target image marked as a normal target image, and saving the reconstructed target image to the forward sample library; when the target image is marked as a pending target image, calculating an image similarity between the target image and the forward sample image, and determining whether the target image is an abnormal image according to the image similarity; if it is an abnormal image, obtaining an abnormal sample image corresponding to the shooting point in an abnormal sample library, calculating a difference value between the target image and the abnormal sample image, and when it is determined that the shooting point is an abnormal inspection point according to the difference value, performing image reconstruction on the target image, and saving the reconstructed target image to the abnormal sample library; the method further comprises: receiving a pre-detection image shot by an inspection device at a pre-prepared shooting point, identifying the pre-detection image, obtaining pixel values of each pixel point in the pre-detection image, calculating quality data of the pre-detection image according to the pixel values, wherein the quality data comprises gradient information and image peak signal-to-noise ratio; if the quality data is greater than or equal to a preset quality threshold, denoising the pre-detection image and determining that the denoised pre-detection image is a target image; if the quality data is less than the preset quality threshold, generating a secondary shooting instruction according to the shooting point of the pre-detection image, shooting a new pre-detection image, until a target image satisfying the preset quality threshold is obtained; the method further comprises: performing subtraction operation on the pixel values of each pixel point in the target image and the pixel values of the pixel points at the corresponding positions in the forward sample image; according to the operation result, marking the target image as a normal target image or a pending target image.

2. The method of claim 1, wherein, after obtaining the target image of the substation equipment, the method further comprises: calculating a scale space of the target image according to a horizontal coordinate, a vertical coordinate, a standard deviation of a normal distribution, and a preset scale space factor of the target image; constructing a Gaussian pyramid based on the scale space and performing Gaussian blur operation to obtain corner point information of the target image; projecting the target image onto the forward sample image according to the corner point information, and aligning the target image with the forward sample image.

3. The method of claim 1, wherein, the method further comprises: The high-dimensional feature information of the target image is recognized through a convolutional neural network, positive feature information is generated based on the high-dimensional feature information, and a positive reconstruction image is constructed according to the positive feature information, and the positive reconstruction image is saved as a positive sample image of the shooting point in the positive sample library.

4. The method of claim 1, wherein, The image similarity is used to determine whether the target image is an abnormal image, including: If the image similarity is less than or equal to a preset similarity threshold, the target image is determined to be a normal target image, and the target image is saved to the positive sample library; If the image similarity is greater than the preset similarity threshold, the target image is determined to be an abnormal image.

5. The method of claim 1, wherein, When the difference value is less than or equal to a preset difference value, the shooting point is determined to be an abnormal inspection point, abnormal inspection information of the shooting point is generated according to the target image, abnormal inspection information alarm is performed, and the target image is reconstructed, and the reconstructed target image is saved to the abnormal sample library. When the image difference value is greater than the preset difference value, the target image is sent to an operation and maintenance terminal, so that an operation and maintenance personnel performs abnormal review on the target image, and the target image is reconstructed according to the received abnormal review result, and the reconstructed target image is saved to the positive sample library or the abnormal sample library. The device includes:

6. A substation anomaly inspection device characterized by comprising: A first acquisition module is configured to acquire a target image of a substation equipment in response to a substation equipment inspection task instruction; A second acquisition module is configured to acquire a positive sample image corresponding to the target image from a positive sample library according to a shooting point of the target image; A marking module is configured to mark the target image as a normal target image or a pending target image according to pixel information of the target image and pixel information of the positive sample image, and to reconstruct the target image marked as the normal target image when the target image is marked as the normal target image, and save the reconstructed target image to the positive sample library; A judgment module is configured to calculate an image similarity between the target image and the positive sample image when the target image is marked as the pending target image, and to determine whether the target image is an abnormal image according to the image similarity; A determination module is configured to acquire an abnormal sample image corresponding to the shooting point from an abnormal sample library if the target image is the abnormal image, to calculate a difference value between the target image and the abnormal sample image, and to reconstruct the target image if the shooting point is determined to be an abnormal inspection point according to the difference value, and save the reconstructed target image to the abnormal sample library. The first acquisition module is further configured to: ​ The receiving inspection device receives a pre-detection image taken at a pre-prepared shooting point, identifies the pre-detection image, obtains pixel values of each pixel point in the pre-detection image, and calculates quality data of the pre-detection image according to the pixel values, wherein the quality data comprises gradient information and an image peak signal-to-noise ratio; If the quality data is greater than or equal to a preset quality threshold, the pre-detection image is denoised, and a denoised pre-detection image is determined as a target image; If the quality data is less than the preset quality threshold, a secondary shooting instruction is generated according to a shooting point of the pre-detection image, a new pre-detection image is taken, and the process is repeated until a target image satisfying the preset quality threshold is obtained; The marking module is further configured to: Subtract the pixel values of each pixel point in the target image from the pixel values of the corresponding pixel points in the forward sample image; According to the operation result, the target image is marked as a normal target image or a pending target image.

7. A storage medium having stored thereon a computer program, characterized in that The computer program, when executed by a processor, implements the method for substation anomaly inspection according to any one of claims 1 to 5.

8. A computer device comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, characterized in that, The processor, when executing the computer program, implements the method for substation anomaly inspection according to any one of claims 1 to 5.

Images