Substation defect identification method and device, medium and equipment

Abstract

The invention discloses a substation defect identification method and device, a medium and equipment. The method comprises the following steps: identifying inspection picture data in an inspection task, and determining an abnormal inspection picture data set; searching a normal sample data set with the highest similarity with the abnormal inspection picture data set from a normal sample database; registering inspection pictures in the abnormal inspection picture data set with pictures in the corresponding normal sample data set, and determining a registered abnormal inspection picture data set; detecting a region of interest in the normal sample data set by using an equipment detection model, and performing global change detection on the registration anomaly inspection picture data set and the normal sample data set by using a change detection model to determine a predicted change region; and rejecting an invalid change detection result of the prediction change region by using the region of interest, determining an abnormal detection result of the abnormal inspection picture data set, and determining a defect identification result of the transformer substation according to the abnormal detection result.

Description

Technical Field

[0001] The present invention relates to the technical field of power equipment defect identification, and more specifically, to a substation defect identification method, device, medium and equipment. Background Art

[0002] With the continuous development of social economy, the scale of power grid construction has grown rapidly, and the business volume of daily inspection and operation has increased rapidly, resulting in a significant increase in the workload, labor intensity and pressure of the existing operation and inspection team. Intelligent and digital technologies are urgently needed to help reduce the burden on the grassroots front line. With the development of intelligent technology, substation operation and maintenance operations have gradually introduced automated inspection systems to improve detection efficiency and accuracy. However, current technologies face the following three key problems in practical applications, which seriously affect the overall performance and practicality of the intelligent inspection system: 1) Scarcity of defect data leads to low model detection rate. During the inspection of substations, defect samples are usually far less than normal samples. If these scarce defect samples are directly used to train the defect detection model, the model detection rate will be significantly reduced. This is because the model cannot fully learn the characteristics of the defects during training, resulting in insufficient defect detection capabilities, which in turn affects the reliability and safety of the system.

[0003] 2) Too many normal samples reduce the efficiency of manual review. The number of normal inspection samples generated during the operation and maintenance of substations is huge, and the processing of these normal samples greatly increases the workload and cost of manual review. The existing manual review method is difficult to cope with the massive amount of normal samples, resulting in low efficiency and a significant increase in work costs.

[0004] 3) The diversity of inspection image styles affects the accuracy of the model. The images taken during actual inspections are affected by many factors such as weather, lighting, seasonal changes, and shooting angles, which makes the image styles diverse. This diversity reduces the accuracy of the defect recognition model in different environments and makes it difficult to maintain style consistency. Summary of the invention

[0005] In view of the deficiencies in the prior art, the present invention provides a substation defect identification method, device, medium and equipment.

[0006] According to one aspect of the present invention, a method for identifying substation defects is provided, comprising: Use the pre-trained abnormal image screening model to identify the inspection image data in the inspection task and determine the abnormal inspection image data set; Use the pre-trained image search model to retrieve the normal sample dataset with the highest similarity to the abnormal inspection image dataset from the normal sample database; Align the inspection pictures in the abnormal inspection picture data set with the pictures in the corresponding normal sample data set to determine the aligned abnormal inspection picture data set; The device detection model is used to detect the region of interest in the normal sample data set, and the change detection model is used to perform global change detection on the registered abnormal inspection image data set and the normal sample data set to determine the predicted change area; The invalid change detection results of the predicted change area are eliminated by using the region of interest, the anomaly detection results of the abnormal inspection image dataset are determined, and the defect identification results of the substation are determined based on the anomaly detection results.

[0007] Optionally, the abnormal image screening model is obtained by training normal samples in a normal sample database, wherein the normal sample database is composed of substation equipment images collected at fixed points.

[0008] Optionally, the total loss function of the abnormal image screening model is: in, L rec is the total loss, L MSE is the mean square error loss, L MSSIM is the structural similarity loss, I r To reconstruct an image based on a normal sample, I ori Original input inspection image, mean square error loss L MSE The expression is: in, n is the total number of pixels, i is the pixel position, y i To reconstruct the image at pixel i The pixel value at is the original input inspection image pixel i The pixel value at ; Structural Similarity Loss L MSSIM The expression is: in, L SSIM Used to measure the structural consistency across the entire image.

[0009] Optionally, structural consistency L SSIM The expression is: in, x To reconstruct the image, y Inspection images, is the average brightness of the image, To reconstruct the image contrast, is the inspection image contrast, is the covariance between the reconstructed image and the inspection image, where: In the formula, w i For the pixel i and the Gaussian kernel weights are 11×11, , , are the mean, contrast, and covariance on a patch, respectively.

[0010] Optionally, a pre-trained image search model is used to retrieve a normal sample data set with the highest similarity to the abnormal inspection image data set from a normal sample database, including: The image search model is used to search the normal sample database of each point in the abnormal inspection picture data set for the normal sample data with the highest similarity to the current abnormal inspection picture in terms of lighting, weather and season to form a normal sample data set.

[0011] Optionally, registering the inspection pictures in the abnormal inspection picture data set with pictures in the corresponding normal sample data set to determine the registered abnormal inspection picture data set includes: Use deep learning models to extract inspection feature points and inspection feature descriptors of abnormal inspection images in the abnormal inspection image dataset; Use a deep learning model to extract normal feature points and normal feature descriptors of normal samples in a normal sample dataset; According to the inspection feature points, inspection feature descriptors, normal feature points and normal feature descriptors, brute force rough matching is used to perform feature matching on the corresponding abnormal inspection pictures and normal samples to obtain rough matching results; Use the ransac algorithm to eliminate the mismatch results in the rough matching results and obtain the precise matching results; Find the optimal homography matrix in the precise matching result, and use the homography matrix to perform perspective transformation on the abnormal inspection pictures in the abnormal inspection picture data set to obtain the registered abnormal inspection picture data after registration.

[0012] Optionally, the change detection model uses a U-Net structure multi-scale feature encoder-decoder and a resnet-50 structure encoder, and adds an attention mechanism, where the loss function of the change detection model is F 1- Score for: in, In the formula, Precision is the accuracy, Recall is the recall rate, TP For real example, FP For false positive examples, FN is a false negative example.

[0013] According to another aspect of the present invention, there is provided a substation defect identification device, comprising: The recognition module is used to use a pre-trained abnormal image screening model to identify the inspection image data in the inspection task and determine the abnormal inspection image data set; A retrieval module is used to retrieve a normal sample data set with the highest similarity to the abnormal inspection image data set from a normal sample database using a pre-trained image search model; A registration module is used to register the inspection pictures in the abnormal inspection picture data set with the pictures in the corresponding normal sample data set to determine the registered abnormal inspection picture data set; The detection module is used to detect the region of interest in the normal sample data set using the device detection model, and to perform global change detection on the registered abnormal inspection image data set and the normal sample data set using the change detection model to determine the predicted change region; The determination module is used to use the region of interest to eliminate invalid change detection results of the predicted change area, determine the anomaly detection results of the abnormal inspection image data set, and determine the defect identification results of the substation based on the anomaly detection results.

[0014] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the storage medium stores a computer program, and the computer program is used to execute the method described in any one of the above aspects of the present invention.

[0015] According to another aspect of the present invention, an electronic device is provided, comprising: a processor; a memory for storing instructions executable by the processor; the processor is configured to read the executable instructions from the memory and execute the instructions to implement the method described in any one of the above aspects of the present invention.

[0016] Therefore, the substation defect identification method proposed by the present invention will effectively improve the accuracy and detection rate of defect identification in the substation intelligent patrol system task, greatly reduce the cost of manual review, and improve the efficiency of manual review. The abnormal image screening model is used to eliminate a large number of normal samples in the patrol task, and the abnormal samples are preliminarily determined in advance, which can greatly reduce the workload of the review personnel. The early locking of abnormal samples also reduces the missed detection rate of substation defect identification. The image search model is used to find the normal sample with the highest similarity to the current patrol image in terms of lighting, weather, and season in the normal sample library, which can eliminate the influence of lighting, weather, and season on subsequent defect identification. The patrol image is aligned with the samples in the retrieved normal sample library to eliminate the errors introduced by different shooting angles. The detection capability of the change detection model is used to perform global change detection on the patrol samples and normal samples. The ROI area is extracted by the equipment detection model, and invalid change detection results are eliminated by the ROI area. Errors are gradually filtered through each link, thereby improving the accuracy and detection rate of the final defect identification. The present invention can be widely used in all substations with voltage levels of 220kV and above equipped with intelligent patrol systems. By introducing advanced positive and negative sample comparison algorithm strategies, the efficiency and effect of defect identification can be significantly improved, and faster and more accurate defect detection can be achieved. In addition, the application of this method will greatly reduce the patrol cycle, thereby reducing the review workload of operation and maintenance personnel. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] A more complete understanding of exemplary embodiments of the present invention may be obtained by referring to the following drawings: Figure 1 is a flow chart of a method for identifying substation defects provided by an exemplary embodiment of the present invention; Figure 2 It is a flow chart of a normal sample library and an abnormal sample library in a substation defect identification algorithm provided by an exemplary embodiment of the present invention; Figure 3 is a network diagram of a normal sample reconstruction algorithm provided by an exemplary embodiment of the present invention; Figure 4 It is a registration process of inspection images and normal sample images provided by an exemplary embodiment of the present invention; Figure 5 is a schematic diagram of a change detection model provided by an exemplary embodiment of the present invention; Figure 6 is a schematic diagram of change detection result screening provided by an exemplary embodiment of the present invention; Figure 7 is a schematic structural diagram of a substation defect identification device provided by an exemplary embodiment of the present invention; Figure 8 This is a structure of an electronic device provided by an exemplary embodiment of the present invention. DETAILED DESCRIPTION

[0018] Below, the exemplary embodiments according to the present invention will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments of the present invention, and it should be understood that the present invention is not limited to the exemplary embodiments described here.

[0019] It should be noted that the relative arrangement of components and steps, the numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present invention unless specifically stated otherwise.

[0020] Those skilled in the art can understand that the terms "first" and "second" in the embodiments of the present invention are only used to distinguish different steps, devices or modules, etc., and neither represent any specific technical meaning nor indicate the necessary logical order between them.

[0021] It should also be understood that, in the embodiments of the present invention, “plurality” may refer to two or more than two, and “at least one” may refer to one, two or more than two.

[0022] It should also be understood that any component, data or structure mentioned in the embodiments of the present invention can generally be understood as one or more, unless explicitly limited or otherwise indicated in the context.

[0023] In addition, the term "and / or" in the present invention is only a description of the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in the present invention generally indicates that the associated objects before and after are in an "or" relationship.

[0024] It should also be understood that the description of the various embodiments of the present invention focuses on the differences between the various embodiments, and the same or similar aspects thereof can be referenced to each other, and for the sake of brevity, they will not be described one by one.

[0025] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0026] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the invention, its application, or uses.

[0027] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered part of the specification.

[0028] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0029] Embodiments of the present invention can be applied to electronic devices such as terminal devices, computer systems, servers, etc., which can operate with many other general or special computing system environments or configurations. Examples of well-known terminal devices, computing systems, environments and / or configurations suitable for use with electronic devices such as terminal devices, computer systems, servers, etc. include, but are not limited to: personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, network personal computers, small computer systems, large computer systems, and distributed cloud computing technology environments including any of the above systems, etc.

[0030] Electronic devices such as terminal devices, computer systems, servers, etc. can be described in the general context of computer system executable instructions (such as program modules) executed by computer systems. Generally, program modules can include routines, programs, object programs, components, logic, data structures, etc., which perform specific tasks or implement specific abstract data types. Computer systems / servers can be implemented in a distributed cloud computing environment, where tasks are performed by remote processing devices linked through a communication network. In a distributed cloud computing environment, program modules can be located on local or remote computing system storage media including storage devices.

[0031] Exemplary Methods Figure 1 FIG. 1 is a flow chart of a method for identifying substation defects provided by an exemplary embodiment of the present invention. This embodiment can be applied to electronic devices, such as Figure 1 As shown, the substation defect identification method 100 includes the following steps: Step 101, using a pre-trained abnormal image screening model to identify inspection image data in the inspection task, and determine an abnormal inspection image data set; Step 102, using a pre-trained image search model to retrieve a normal sample data set with the highest similarity to the abnormal inspection image data set from a normal sample database; Step 103, registering the inspection pictures in the abnormal inspection picture data set with the pictures in the corresponding normal sample data set to determine the registered abnormal inspection picture data set; Step 104, using the device detection model to detect the region of interest in the normal sample data set, and using the change detection model to perform global change detection on the registered abnormal inspection image data set and the normal sample data set to determine the predicted change region; Step 105, using the region of interest to eliminate invalid change detection results of the predicted change region, determine the abnormal detection results of the abnormal inspection image data set, and determine the defect recognition results of the substation according to the abnormal detection results.

[0032] Specifically, the present invention proposes a substation defect identification method to solve 1) the problem of low model detection rate due to the sample distribution characteristics of less defect data and more normal samples; 2) the problem of low manual review efficiency and high labor costs caused by a large number of normal inspection samples during substation operation and maintenance; 3) the problem of reduced accuracy of defect identification models due to weather, light, season, and shooting angle deviation. By constructing a substation defect identification method, the substation intelligent inspection task can be completed efficiently and accurately. Specifically, it includes the following steps: Step S101, use the pre-trained abnormal image screening model to eliminate a large number of normal samples in the inspection task and preliminarily determine the abnormal samples in advance. Based on this method, in the substation inspection task, a normal sample library and an abnormal sample library will be maintained according to different inspection points. Specifically, the role of the normal sample library and the abnormal sample library in the substation defect identification process is as follows: Figure 2 shown.

[0033] Specifically, in the intelligent inspection task of the substation, the abnormal image screening model is trained based on a large number of normal inspection samples, and the inspection samples are placed in the normal sample library or the abnormal sample library according to the recognition results of the inspection samples in the entire defect identification process. As the normal sample library and the abnormal sample library increase, the normal sample library and the abnormal sample library are cleaned, and the change detection model and the equipment detection model of the general image in step S105 are adjusted using valid data.

[0034] For the abnormal image screening model, specifically, Figure 3 By analyzing an image into features, and then reconstructing it into a normal sample based on the features. By comparing the reconstructed normal sample with the original sample, it is determined whether the original sample is an abnormal image.

[0035] First, the abnormal image screening model is trained using the data in the normal sample library. During the intelligent inspection of the substation, the positive and negative sample attributes of the inspection image are determined by comparing the consistency between the inspection image and the reconstructed image at the pixel level. During the training process, the loss function is constructed by constraining the consistency between the reconstructed sample and the input normal sample. Specifically, the total loss function of the abnormal image screening model is as follows: (1) in, L rec is the total loss, L MSE is the mean square error loss, L MSSIMis the structural similarity loss, I r To reconstruct an image based on a normal sample, I ori Original input inspection image, L MSE The consistency between the reconstructed image and the inspected image is constrained from the perspective of pixel values. L MSE The details are shown in formula (2).

[0036] (2) in, n is the total number of pixels, i is the pixel position, y i To reconstruct the image at pixel i The pixel value at is the original input inspection image pixel i The pixel value at . L SSIM The consistency between the reconstructed image and the input inspection image is constrained in terms of image contrast, brightness and structure. L MSSIM The specific formula is shown in (3).

[0037] (3) in, L SSIM Measures the structural consistency of the entire image. Since the span of the entire image in terms of brightness, contrast, and structure is too large, it is difficult to calculate in practice. L SSIM It should be done on a patch. L MSSIM For M Average over patches L SSIM . L SSIM The specific formula is shown in (4).

[0038] (4) in, x To reconstruct the image, y This is an inspection image. is the average brightness of the image, To reconstruct the image contrast, is the inspection image contrast, is the covariance of the reconstructed image and the inspection image. Assume that the Gaussian kernel weight at the pixel point is 11×11. w i, taking the reconstructed image as an example, the mean value on a patch , contrast , covariance The formulas are shown in (5), (6) and (7) respectively.

[0039] (5) (6) (7) The inspection pictures are identified by the abnormal picture screening model trained by the normal sample library, and the identification results include both normal and abnormal situations. Thus, the abnormal inspection picture data set in the inspection picture data is identified by the abnormal picture screening model.

[0040] Step S102, using an image search model to find the normal sample library of each point shot at a fixed angle, which has the highest similarity with the current inspection picture in the screened abnormal inspection picture data set in terms of lighting, weather, and season, to form a normal sample data set; Specifically, when the abnormal image screening model detects defects in the inspection image, the image search model is used to find the normal sample image that is most similar to the current inspection image in terms of lighting, weather, season, etc. in the normal sample library of the current point, forming a pair of inspection image and normal sample image to prepare for subsequent image defect recognition. At this stage, the influence of lighting, weather, season, etc. on subsequent defect recognition is eliminated.

[0041] Step S103, registering the inspection pictures in the abnormal inspection picture data set with the corresponding normal samples in the retrieved normal sample data set to eliminate errors introduced by different shooting angles; Specifically, a fixed perspective is used to shoot samples at the inspection point, and the pictures taken by the robot during the intelligent inspection of the substation have a certain deviation from the fixed camera in terms of perspective. This shooting perspective deviation will cause large errors in subsequent defect identification. Therefore, an image registration algorithm is needed to correct the errors caused by the different shooting perspectives of the robot.

[0042] The registration process is as follows Figure 4 As shown in the figure. First, the deep learning model is used to extract the inspection image and the retrieval model is used to output the feature points and feature descriptors of the normal sample image from the normal sample library. The feature points and descriptors of the two images are matched using brute force coarse matching. The RANSAC algorithm is used to eliminate the mismatch results and find the optimal homography matrix. The inspection image is transformed in perspective through the homography matrix, and finally the registered inspection image is obtained.

[0043] Step S104, using the detection capability of the change detection model to perform global change detection on the inspection samples and normal samples; Specifically, Figure 5 As shown in the figure, the overall structure of the change detection model adopts a multi-scale feature encoder-decoder similar to the U-Net structure, and adds a temporal attention mechanism. The encoder adopts the resnet-50 structure. In the calculation process of the temporal attention mechanism, the feature map on the feature channel of the inspection point image is used to generate the query matrix. The query matrix is ​​operated by the key matrix, and the inspection point is generated as the covariance between each pixel on the image channel and the related pixels within the dependency range of the inspection image feature channel. The covariance matrix after the softmax operation provides the precise similarity between pixels, and then the value matrix inferred from the feature map of the inspection image feature channel is used for fusion operation to predict the change area.

[0044] The loss function of the network is F The 1-value constraint inspection point is the consistency between the changed area and the label of the image and the inspection image. F The specific form of the value 1 is shown in formula (8).

[0045] (8) in, Precision is the accuracy, Recall is the recall rate. The specific forms of precision and recall rate are shown in formulas (9) and (10).

[0046] (9) (10) in, TP (True Positive, true example), FP (False Positive, false positive example), FN (False Negative) is calculated at the pixel level based on the consistency between the predicted change mask and the true label pixels.

[0047] Step S105, reference Figure 6 As shown, the device detection model is used to extract the ROI area, and invalid change detection results are eliminated through the ROI area.

[0048] Specifically, the change detection model is first used to detect all the change areas in the normal sample image with the inspection image and the inspection points. The device detection model is used to detect the device of interest. This is not limited to the target detection model. The minimum bounding rectangle is generated based on the mask in the global change area. The changes in the area of ​​interest are selected by calculating the IOU area of ​​the change bounding rectangle and the device frame, and other changes are eliminated. This method filters out other interference factors, such as changes in the sky, grass, etc., and finally outputs the change results of the area of ​​interest.

[0049] Therefore, the substation defect identification method proposed in the present invention will effectively improve the accuracy and detection rate of defect identification in the substation intelligent patrol system task, greatly reduce the cost of manual review, and improve the efficiency of manual review. The abnormal image screening model is used to eliminate a large number of normal samples in the patrol task, and the abnormal samples are preliminarily determined in advance, which can greatly reduce the workload of the review personnel. The early locking of abnormal samples also reduces the missed detection rate of substation defect identification. The image search model is used to find the normal sample with the highest similarity to the current patrol image in terms of lighting, weather, and season in the normal sample library, which can eliminate the influence of lighting, weather, and season on subsequent defect identification. The patrol image is aligned with the samples in the retrieved normal sample library to eliminate the errors introduced by different shooting angles. The detection capability of the change detection model is used to perform global change detection on the patrol samples and normal samples. The ROI area is extracted by the equipment detection model, and invalid change detection results are eliminated by the ROI area. Errors are filtered step by step through each link, thereby improving the accuracy and detection rate of the final defect identification. The present invention can be widely used in all substations with voltage levels of 220kV and above equipped with intelligent patrol systems. By introducing advanced positive and negative sample comparison algorithm strategies, the efficiency and effect of defect identification can be significantly improved, and faster and more accurate defect detection can be achieved. In addition, the application of this method will greatly reduce the patrol cycle, thereby reducing the review workload of operation and maintenance personnel.

[0050] Exemplary Devices Figure 7 FIG. 1 is a schematic diagram of the structure of a substation defect identification device provided by an exemplary embodiment of the present invention. Figure 7 As shown, the apparatus 700 includes: The recognition module 710 is used to use a pre-trained abnormal image screening model to recognize the inspection image data in the inspection task and determine the abnormal inspection image data set; Retrieval module 720, used to retrieve the normal sample data set with the highest similarity to the abnormal inspection image data set from the normal sample database using a pre-trained image search model; A registration module 730 is used to register the inspection pictures in the abnormal inspection picture data set with the pictures in the corresponding normal sample data set to determine the registered abnormal inspection picture data set; The detection module 740 is used to detect the region of interest in the normal sample data set using the device detection model, and to perform global change detection on the registered abnormal inspection image data set and the normal sample data set using the change detection model to determine the predicted change region; The determination module 750 is used to use the region of interest to eliminate invalid change detection results of the predicted change area, determine the abnormal detection results of the abnormal inspection image data set, and determine the defect identification results of the substation according to the abnormal detection results.

[0051] Optionally, the abnormal image screening model is obtained by training normal samples in a normal sample database, wherein the normal sample database is composed of substation equipment images collected at fixed points.

[0052] Optionally, the total loss function of the abnormal image screening model is: in, L rec is the total loss, L MSE is the mean square error loss, L MSSIM is the structural similarity loss, I r To reconstruct an image based on a normal sample, I ori Original input inspection image, mean square error loss L MSE The expression is: in, n is the total number of pixels, i is the pixel position, y i To reconstruct the image at pixel i The pixel value at is the original input inspection image pixel i The pixel value at ; Structural Similarity Loss L MSSIM The expression is: in, L SSIM Used to measure the structural consistency across the entire image.

[0053] Optionally, structural consistency L SSIM The expression is: in, x To reconstruct the image, y Inspection images, is the average brightness of the image, To reconstruct the image contrast, is the inspection image contrast, is the covariance between the reconstructed image and the inspection image, where: In the formula, w i For the pixel i and the Gaussian kernel weights are 11×11, , , are the mean, contrast, and covariance on a patch, respectively.

[0054] Optionally, the retrieval module 720 includes: The search submodule is used to use the image search model to search for the normal sample data with the highest similarity to the current abnormal inspection picture in terms of lighting, weather and season in the normal sample database of each point in the abnormal inspection picture data set to form a normal sample data set.

[0055] Optionally, the registration module 730 includes: The first extraction submodule is used to extract inspection feature points and inspection feature descriptors of abnormal inspection pictures in the abnormal inspection picture dataset using a deep learning model; A second extraction submodule is used to extract normal feature points and normal feature descriptors of normal samples in a normal sample data set using a deep learning model; The coarse matching submodule is used to perform feature matching on the corresponding abnormal inspection pictures and normal samples using brute force coarse matching based on the inspection feature points, inspection feature descriptors, normal feature points and normal feature descriptors to obtain coarse matching results; The fine matching submodule is used to use the ransac algorithm to eliminate the false matching results in the rough matching results and obtain the fine matching results; The registration submodule is used to find the optimal homography matrix in the precise matching result, and perform perspective transformation on the abnormal inspection picture in the abnormal inspection picture data set through the homography matrix to obtain the registered abnormal inspection picture data.

[0056] Optionally, the change detection model uses a U-Net structure multi-scale feature encoder-decoder and a resnet-50 structure encoder, and adds an attention mechanism, where the loss function of the change detection model is F 1- Score for: in, In the formula, Precision is the accuracy, Recall is the recall rate, TP For real example, FP For false positive examples, FN is a false negative example.

[0057] Exemplary Electronic Devices Figure 8 This is a structure of an electronic device provided by an exemplary embodiment of the present invention. Figure 8 As shown, the electronic device 80 includes one or more processors 81 and a memory 82 .

[0058] The processor 81 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device to perform desired functions.

[0059] The memory 82 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, a random access memory (RAM) and / or a cache memory (cache), etc. The non-volatile memory may include, for example, a read-only memory (ROM), a hard disk, a flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 81 may run the program instructions to implement the methods of the software programs of the various embodiments of the present invention described above and / or other desired functions. In one example, the electronic device may also include: an input device 83 and an output device 84, which are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0060] In addition, the input device 83 may also include, for example, a keyboard, a mouse, etc.

[0061] The output device 84 can output various information to the outside, and can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto.

[0062] Of course, to simplify, Figure 8 Only some of the components related to the present invention in the electronic device are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, the electronic device may further include any other appropriate components according to specific application conditions.

[0063] Exemplary computer program products and computer-readable storage media In addition to the above-mentioned methods and devices, an embodiment of the present invention may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the method according to various embodiments of the present invention described in the above-mentioned "Exemplary Method" section of this specification.

[0064] The computer program product may be written in any combination of one or more programming languages ​​to write program code for performing the operations of the embodiments of the present invention, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0065] In addition, an embodiment of the present invention may also be a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, enable the processor to execute the steps of the method according to various embodiments of the present invention described in the above “Exemplary Method” section of this specification.

[0066] The computer readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, system or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0067] The basic principle of the present invention is described above in conjunction with specific embodiments. However, it should be pointed out that the advantages, strengths, effects, etc. mentioned in the present invention are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. must be possessed by each embodiment of the present invention. In addition, the specific details disclosed above are only for the purpose of illustration and facilitation of understanding, rather than limitation, and the above details do not limit the present invention to being implemented by adopting the above specific details.

[0068] Each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system embodiment, since it basically corresponds to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0069] The block diagrams of the devices, systems, equipment, and systems involved in the present invention are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagram. As will be appreciated by those skilled in the art, these devices, systems, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including", "comprising", "having", etc. are open words, referring to "including but not limited to", and can be used interchangeably with them. The words "or" and "and" used here refer to the words "and / or" and can be used interchangeably with them, unless the context clearly indicates otherwise. The word "such as" used here refers to the phrase "such as but not limited to", and can be used interchangeably with it.

[0070] The method and system of the present invention may be implemented in many ways. For example, the method and system of the present invention may be implemented by software, hardware, firmware or any combination of software, hardware, firmware. The above order of steps for the method is only for illustration, and the steps of the method of the present invention are not limited to the order specifically described above, unless otherwise specifically stated. In addition, in some embodiments, the present invention may also be implemented as a program recorded in a recording medium, which includes machine-readable instructions for implementing the method according to the present invention. Thus, the present invention also covers a recording medium storing a program for executing the method according to the present invention.

[0071] It should also be noted that in the system, device and method of the present invention, each component or each step can be decomposed and / or recombined. These decompositions and / or recombinations should be regarded as equivalent schemes of the present invention. The above description of the disclosed aspects is provided to enable any technician in the field to make or use the present invention. Various modifications to these aspects are very obvious to those skilled in the art, and the general principles defined here can be applied to other aspects without departing from the scope of the present invention. Therefore, the present invention is not intended to be limited to the aspects shown here, but in accordance with the widest range consistent with the principles and novel features disclosed here.

[0072] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present invention to the forms disclosed herein. Although a number of example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims

1. A method for identifying substation defects, characterized in that: include: Use the pre-trained abnormal image screening model to identify the inspection image data in the inspection task and determine the abnormal inspection image data set; Retrieving a normal sample data set having the highest similarity to the abnormal inspection image data set from a normal sample database using a pre-trained image search model; Aligning the inspection pictures in the abnormal inspection picture data set with the pictures in the corresponding normal sample data set to determine the aligned abnormal inspection picture data set; Detecting the region of interest in the normal sample data set using the device detection model, and performing global change detection on the registered abnormal inspection image data set and the normal sample data set using the change detection model to determine the predicted change region; The invalid change detection results of the predicted change area are eliminated by using the region of interest to determine the abnormal detection results of the abnormal inspection image data set, and the defect identification results of the substation are determined according to the abnormal detection results.

2. The method according to claim 1, characterized in that The abnormal image screening model is obtained by training normal samples in a normal sample database, wherein the normal sample database is composed of substation equipment images collected at fixed points.

3. The method according to claim 1 or 2, characterized in that: The total loss function of the abnormal image screening model is: in, L rec is the total loss, L MSE is the mean square error loss, L MSSIM is the structural similarity loss, I r To reconstruct an image based on a normal sample, I ori Original input inspection image, mean square error loss L MSE The expression is: in, n is the total number of pixels, i is the pixel position, y i To reconstruct the image at pixel i The pixel value at is the original input inspection image pixel i The pixel value at ; Structural Similarity Loss L MSSIM The expression is: in, L SSIM Used to measure the structural consistency across the entire image.

4. The method according to claim 3, characterized in that The structural consistency L SSIM The expression is: in, x To reconstruct the image, y Inspection images, is the average brightness of the image, To reconstruct the image contrast, is the inspection image contrast, is the covariance between the reconstructed image and the inspection image, where: In the formula, w i For the pixel i and the Gaussian kernel weights are 11×11, , , are the mean, contrast, and covariance on a patch, respectively.

5. The method according to claim 1, characterized in that The normal sample data set with the highest similarity to the abnormal inspection image data set is retrieved from the normal sample database using the pre-trained image search model, including: The image search model is used to search the normal sample database of each point in the abnormal inspection picture data set for normal sample data with the highest similarity to the current abnormal inspection picture in terms of lighting, weather and season, to form the normal sample data set.

6. The method according to claim 1, characterized in that The inspection pictures in the abnormal inspection picture data set are registered with pictures in the corresponding normal sample data set to determine the registered abnormal inspection picture data set, including: Extracting inspection feature points and inspection feature descriptors of abnormal inspection pictures in the abnormal inspection picture dataset using a deep learning model; Extracting normal feature points and normal feature descriptors of normal samples in the normal sample data set using a deep learning model; According to the inspection feature points, the inspection feature descriptors, the normal feature points and the normal feature descriptors, using brute force coarse matching to perform feature matching on the corresponding abnormal inspection pictures and normal samples to obtain a coarse matching result; Use the RANSAC algorithm to eliminate the mismatched results in the rough matching results to obtain the precise matching results; An optimal homography matrix in the precise matching result is found, and a perspective transformation is performed on the abnormal inspection picture in the abnormal inspection picture data set by using the homography matrix to obtain registered abnormal inspection picture data after registration.

7. The method according to claim 1, characterized in that The change detection model adopts a U-Net structure multi-scale feature encoder-decoder and a resnet-50 structure encoder, and adds an attention mechanism, wherein the loss function of the change detection model is F 1- Score for: in, In the formula, Precision is the accuracy, Recall is the recall rate, TP For real example, FP For false positive examples, FN is a false negative example.

8. A substation defect identification device, characterized in that: include: The recognition module is used to use a pre-trained abnormal image screening model to identify the inspection image data in the inspection task and determine the abnormal inspection image data set; A retrieval module, used to retrieve a normal sample data set having the highest similarity with the abnormal inspection image data set from a normal sample database using a pre-trained image search model; A registration module, used to register the inspection pictures in the abnormal inspection picture data set with the pictures in the corresponding normal sample data set, and determine the registered abnormal inspection picture data set; A detection module, used to detect the region of interest in the normal sample data set using the device detection model, and to perform global change detection on the registered abnormal inspection image data set and the normal sample data set using the change detection model to determine the predicted change region; A determination module is used to use the region of interest to eliminate invalid change detection results of the predicted change region, determine the abnormal detection results of the abnormal inspection image data set, and determine the defect identification results of the substation according to the abnormal detection results.

9. The device according to claim 8, characterized in that The abnormal image screening model is obtained by training normal samples in a normal sample database, wherein the normal sample database is composed of substation equipment images collected at fixed points.

10. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and the computer program is used to execute the method according to any one of claims 1 to 7.

11. An electronic device, characterized in that: The electronic device comprises: processor; a memory for storing instructions executable by the processor; The processor is used to read the executable instructions from the memory and execute the instructions to implement the method described in any one of claims 1 to 7.

Images