Multidimensional image front-end analysis method and system suitable for transformer equipment oil leakage detection

Abstract

The present disclosure provides a multi-dimensional image front-end analysis method and system suitable for oil leakage detection of power transformation equipment, image data and environmental perception data of the equipment to be detected are obtained; the image data of the equipment to be detected at least includes: equipment top image, equipment middle image and equipment ground image, and the environmental perception data at least includes equipment infrared detection image data and equipment temperature detection data; the obtained image data and environmental perception data of the equipment to be detected are respectively identified by using a preset neural network model, and the final identification result of the equipment oil leakage defect is obtained by comprehensively analyzing each identification result; the present disclosure performs multi-dimensional image analysis on the oil carrying equipment of the substation based on multi-dimensional stereoscopic image, judges the oil leakage of the equipment in combination with the surrounding multi-dimensional environmental information, and simultaneously optimizes the operation system of the substation inspection robot, thereby improving the identification performance of the substation inspection robot on the oil leakage defect of the equipment.

Description

Technical Field

[0001] This disclosure relates to the field of pattern recognition technology, and in particular to a multi-dimensional image front-end analysis method and system suitable for detecting oil leaks in power equipment. Background Technology

[0002] The statements in this section are merely background information relating to this disclosure and do not necessarily constitute prior art.

[0003] Oil leakage in substation equipment is a serious defect. The inventors have discovered that current substation inspections for identifying oil leakage defects mainly suffer from the following problems:

[0004] (1) The characteristics of oil leakage equipment in substations are different, and the algorithm is prone to misjudgment. The equipment in substations are different in size, oil leakage location, oil leakage speed, and oil color, and are similar to the characteristics of other equipment or defects, such as flashing dirt or paint peeling on the equipment surface. The diversity of characteristics and similarity to other equipment and defects make it impossible for the current mainstream algorithms to effectively and accurately identify oil leakage defects.

[0005] (2) Substation inspection robots lack sufficient computing power to perform complex algorithms. Currently, substation inspection methods mainly combine dynamic inspection by substation inspection robots with static inspection by fixed cameras. Fixed cameras rely on backend servers and have high computing power, but they cannot effectively identify and provide timely warnings of hidden oil leaks; substation inspection robots can inspect hidden areas, but their computing power is limited, making them unable to perform complex algorithm calculations, resulting in low algorithm accuracy. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this disclosure provides a multi-dimensional image front-end analysis method and system suitable for detecting oil leaks in power equipment. Based on multi-dimensional stereo images, it performs multi-dimensional image analysis on oil-bearing equipment in substations, combines surrounding multi-dimensional environmental information to determine oil leaks, and simultaneously optimizes the substation inspection robot's computing system to improve the robot's ability to identify oil leak defects.

[0007] To achieve the above objectives, the present disclosure adopts the following technical solution:

[0008] The first aspect of this disclosure provides a multi-dimensional image front-end analysis method and system suitable for detecting oil leaks in power equipment.

[0009] Acquire image data and environmental perception data of the device under test;

[0010] The image data of the device under test shall include at least: top image of the device, middle image of the device, and ground image of the device; the environmental perception data shall include at least infrared detection image data of the device and temperature detection data of the device.

[0011] Using a pre-set neural network model, the image data and environmental perception data of the device to be tested are processed for identification. By combining the identification results, the final identification result of the oil leakage defect of the device is obtained.

[0012] As an optional implementation, a preset neural network model is used to perform recognition processing on the obtained image data and environmental perception data of the device to be detected, including the following steps:

[0013] For images of the top and middle of the device, the Cascade R-CNN algorithm is used as the backbone network, and an RPN layer is added between the convolutional and pooling layers to remove the background and obtain device features.

[0014] By combining the feature extraction results of the top and middle images of the equipment, the external status information of the equipment is obtained, and the presence of oil seepage on the equipment surface is analyzed based on the external status information.

[0015] Furthermore, using a pre-defined neural network model, the obtained image data and environmental perception data of the device to be detected are processed for identification, including the following steps:

[0016] Based on the acquired ground images of the equipment, an R-CNN neural network is used to obtain ground state information, and then to obtain the ground oil seepage identification result;

[0017] Based on the acquired infrared detection image data, an R-CNN network is used to identify the abnormal infrared regions of the equipment. Combined with the external status information of the equipment, the distance between the abnormal infrared regions and the oil storage points of the equipment is determined to determine whether the problem is caused by oil leakage.

[0018] Furthermore, by synthesizing the various identification results, the final identification result of the equipment oil leakage defect is obtained, including the following steps:

[0019] If the overall temperature of the equipment is abnormal and there is oil seepage on the ground, it is determined that there is an oil leak problem. The specific location of the oil leak is determined by comprehensively judging the abnormal infrared area of ​​the equipment and the external status information of the equipment.

[0020] The second aspect of this disclosure provides a multi-dimensional image front-end analysis system suitable for oil leakage detection in power equipment, comprising:

[0021] The data acquisition module is configured to acquire image data and environmental perception data of the device under test; the image data of the device under test includes at least: top image of the device, middle image of the device, and ground image of the device; the environmental perception data includes at least infrared detection image data and temperature detection data of the device.

[0022] The oil leak detection module is configured to: use a preset neural network model to process the image data and environmental perception data of the device to be detected, and combine the various detection results to obtain the final detection result of the oil leak defect of the device.

[0023] The third aspect of this disclosure provides a multi-dimensional image front-end analysis system suitable for detecting oil leaks in power equipment.

[0024] A multi-dimensional image front-end analysis system for detecting oil leaks in power equipment includes:

[0025] Multiple computational logic units are provided, each of which is configured to run in parallel, and different computational logic units are configured to perform computational tasks of different network layers in the multidimensional image front-end analysis method for detecting oil leakage in power equipment as described in the first aspect of this disclosure.

[0026] The temperature sensing and acquisition unit is used to acquire and process the temperature information of the equipment, and finally feed back abnormal information to the result output unit.

[0027] The data processing unit is configured to preprocess the collected data for oil leakage defects in substation equipment and send it to the corresponding arithmetic logic unit.

[0028] The result output unit is configured to integrate the computational data from each arithmetic logic unit and output the final computation result.

[0029] As an optional implementation, the arithmetic logic unit includes:

[0030] The convolutional activation layer operation unit is configured to perform operations on the convolutional and activation layers in the model and output the operation results;

[0031] The pooling layer computation unit is configured to perform operations on the pooling layers in the model and output the computation results.

[0032] The fully connected layer and output layer computation units are configured to perform computations on the fully connected layers and output layers in the model and output the computation results.

[0033] Redundant computation units are configured to perform computations on other network computation layers and output the results.

[0034] As an optional implementation, the computational tasks of each arithmetic logic unit are executed in parallel.

[0035] The fourth aspect of this disclosure provides a robot, including the multi-dimensional image front-end analysis system for detecting oil leaks in power equipment as described in the second aspect of this disclosure.

[0036] The fifth aspect of this disclosure provides a computer-readable storage medium storing a plurality of instructions adapted for loading by a processor of a terminal device and executing the steps of the multidimensional image front-end analysis method for detecting oil leaks in power equipment as described in the first aspect of this disclosure.

[0037] The sixth aspect of this disclosure provides a terminal device, characterized in that it includes a processor and a computer-readable storage medium, the processor being used to implement various instructions; the computer-readable storage medium being used to store multiple instructions, the instructions being adapted to be loaded by the processor and executed by the processor to perform the steps of the multi-dimensional image front-end analysis method for detecting oil leakage in power equipment as described in the first aspect of this disclosure.

[0038] Compared with the prior art, the beneficial effects of this disclosure are:

[0039] 1. This disclosure innovatively proposes a multi-dimensional image front-end analysis method suitable for oil leakage detection of power equipment. Based on multi-dimensional stereo images, it performs multi-dimensional image analysis on oil-bearing equipment in substations, and combines surrounding multi-dimensional environmental information to determine whether the equipment is leaking oil, thereby improving the identification accuracy of the oil leakage part of the equipment.

[0040] 2. This invention innovatively proposes a multi-dimensional image front-end analysis system suitable for oil leakage detection in power equipment. Through a parallel inference operation mechanism, different structures of the same model are split into different computing units for calculation, which can realize the parallel operation of each computing unit. Compared with the currently commonly used serial operation of the same model, it can achieve a faster running speed.

[0041] Advantages of this disclosure in additional aspects will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this disclosure. Attached Figure Description

[0042] The accompanying drawings, which form part of this disclosure, are used to provide a further understanding of this disclosure. The illustrative embodiments of this disclosure and their descriptions are used to explain this disclosure and do not constitute an undue limitation of this disclosure.

[0043] Figure 1 This is a flowchart illustrating the method for identifying oil leakage defects in substation equipment based on multidimensional images, as provided in Embodiment 1 of this disclosure. Detailed Implementation

[0044] The present disclosure will be further described below with reference to the accompanying drawings and embodiments.

[0045] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of this disclosure. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains.

[0046] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments according to this disclosure. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms “comprising” and / or “including” are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0047] Where there is no conflict, the embodiments and features described herein can be combined with each other.

[0048] Example 1:

[0049] like Figure 1 As shown, Embodiment 1 of this disclosure provides a multi-dimensional image front-end analysis method suitable for oil leakage detection in power equipment, including the following steps:

[0050] S1: The substation inspection robot comprehensively collects information about the surrounding environment of the equipment, including: top image data of the equipment, middle image data of the equipment, ground image data of the equipment, infrared image data of the equipment, and temperature data of the equipment, which are used to input information on whether the equipment is leaking oil.

[0051] S2: For the top and middle images of the device, the Cascade R-CNN algorithm is used as the backbone network, specifically including the following:

[0052] Using R-CNN detectors as the base network, each R-CNN detector generates approximately 2000 candidate regions from the input image. Features are extracted through convolution calculations, and the features are fed into each SVM classifier. Finally, a regressor is used to correct the position of the candidate boxes.

[0053] In this embodiment, three cascaded R-CNN detectors are trained simultaneously. Each cascaded detector is set with a different IOU (Intersection over Union). The output of the current network is used as the input of the next higher-precision network to further improve the network accuracy.

[0054] In this embodiment, an RPN layer is added between the convolutional layer and the pooling layer for background removal and acquisition of prominent features of the device. By combining the two images, external state information of the device is obtained, which is used to analyze whether there is oil seepage on the surface of the device.

[0055] S3: For ground image data of the equipment, an R-CNN network is used to generate about 2,000 candidate regions from the input image. Features are extracted through convolution calculation, and the features are fed into each SVM classifier. Finally, a regressor is used to correct the position of the candidate box. Combined with threshold segmentation, ground state information is obtained to analyze whether there is oil seepage on the ground.

[0056] S4: Infrared image data of the equipment. Using R-CNN network, approximately 2000 candidate regions generated from the input image are processed by convolution to extract features. The features are then fed into each SVM classifier. Finally, a regressor is used to correct the position of the candidate boxes to obtain abnormal areas of the equipment's infrared information. The main analysis focuses on the distance to the equipment's oil storage point to determine whether the leak is caused by the equipment's oil leakage.

[0057] S5: Overall equipment temperature information, obtained by temperature sensors, mainly analyzes whether the current equipment temperature is within the normal range, and uses this as one of the parameters to determine whether oil leakage has occurred;

[0058] S6: Combining S1-S5, if the overall equipment temperature is abnormal and there is oil seepage on the ground, it indicates that there is an oil leak problem. Based on the obtained infrared information, determine the location of the infrared information anomaly, and match the infrared anomaly location with the external feature information of the equipment to obtain the specific oil leak location.

[0059] Example 2:

[0060] Embodiment 2 of this disclosure provides a multi-dimensional image front-end analysis system suitable for oil leakage detection in power equipment, including:

[0061] The data acquisition module is configured to acquire image data and environmental perception data of the device under test; the image data of the device under test includes at least: top image of the device, middle image of the device, and ground image of the device; the environmental perception data includes at least infrared detection image data and temperature detection data of the device.

[0062] The oil leak detection module is configured to: use a preset neural network model to process the image data and environmental perception data of the device to be detected, and combine the various detection results to obtain the final detection result of the oil leak defect of the device.

[0063] The working method of the system is the same as the multi-dimensional image front-end analysis method for detecting oil leakage in power equipment provided in Example 1, and will not be repeated here.

[0064] Example 3:

[0065] Embodiment 3 of this disclosure provides a multi-dimensional image front-end analysis system suitable for oil leakage detection in substation equipment. As can be seen from the multi-dimensional image-based substation equipment oil leakage defect identification method described in Embodiment 1, oil leakage defect identification requires simultaneous analysis of at least four image information and two sensor information. Currently, the computing system of substation inspection robots cannot perform efficient computation. Therefore, the composition of their computing system is optimized to improve their computing power, mainly including:

[0066] Multiple computational logic units are configured to run in parallel, and different computational logic units are configured to perform computational tasks for different network layers in a substation equipment oil leakage defect model for multidimensional images.

[0067] The temperature sensing and acquisition unit is used to acquire and process equipment temperature information, and finally feeds back abnormal information to the result output unit.

[0068] The data processing unit is configured to preprocess the acquired image data and send it to the corresponding arithmetic logic unit;

[0069] The result output unit is configured to integrate the computational data from each arithmetic logic unit and output the final computation result.

[0070] As an alternative implementation, the arithmetic logic unit includes:

[0071] The convolutional activation layer operation unit is configured to perform operations on the convolutional and activation layers in the model and output the operation results;

[0072] The pooling layer computation unit is configured to perform operations on the pooling layers in the model and output the computation results.

[0073] The fully connected layer and output layer computation units are configured to perform computations on the fully connected layers and output layers in the model and output the computation results.

[0074] The threshold segmentation operation unit is used to perform threshold segmentation on the data and output the operation results;

[0075] Redundant computation units are configured to perform computations on other network computation layers and output the results.

[0076] As an alternative implementation, the computational tasks of each arithmetic logic unit are executed in parallel.

[0077] Example 4:

[0078] Embodiment 4 of this disclosure provides a robot, characterized in that it includes a multi-dimensional image front-end analysis system for detecting oil leaks in power equipment as described in Embodiment 2 of this disclosure.

[0079] Example 5:

[0080] Embodiment 5 of this disclosure provides a computer-readable storage medium, characterized in that: it stores a plurality of instructions, which are adapted to be loaded by a processor of a terminal device and execute the steps of the front-end identification method for oil leakage defects in substation equipment based on multi-dimensional images as described in Embodiment 1 of this disclosure.

[0081] Example 6:

[0082] Embodiment 6 of this disclosure provides a terminal device, characterized in that it includes a processor and a computer-readable storage medium, wherein the processor is used to implement various instructions; the computer-readable storage medium is used to store multiple instructions, which are adapted to be loaded by the processor and executed by the processor to perform the steps of the multi-dimensional image front-end analysis method for detecting oil leakage in power equipment as described in Embodiment 1 of this disclosure.

[0083] Those skilled in the art will understand that embodiments of this disclosure can be provided as methods, systems, or computer program products. Therefore, this disclosure can take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware aspects. Furthermore, this disclosure can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0084] This disclosure is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0085] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0086] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0087] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0088] The above description is merely a preferred embodiment of this disclosure and is not intended to limit this disclosure. Various modifications and variations can be made to this disclosure by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A multi-dimensional image front-end analysis method suitable for transformer oil leakage detection, characterized in that: The method comprises the following steps: obtaining image data and environment perception data of a device to be detected; the image data of the device to be detected at least comprises a device top image, a device middle image and a device ground image, and the environment perception data at least comprises device infrared detection image data and device temperature detection data; using a preset neural network model to respectively identify the obtained image data and environment perception data of the device to be detected, and comprehensively analyzing various identification results to obtain a final identification result of a device oil leakage defect; comprehensively analyzing various identification results to obtain a final identification result of a device oil leakage defect, which comprises the following steps: if the device temperature is abnormal as a whole and there is oil seepage on the ground, it is determined that the device has an oil leakage problem, and a specific oil leakage position is determined according to the obtained device infrared abnormal area and device external state information; based on multi-dimensional stereoscopic images, a multi-dimensional image analysis is performed on the oil-bearing equipment in the substation, combined with the surrounding multi-dimensional environment information, whether the device leaks oil is determined, and three cascaded R-CNN detectors are trained, each cascaded detector sets different IOU, and the output of the current network is used as the input of the next higher-precision network.

2. The multi-dimensional image front-end analysis method suitable for oil leakage detection of power transformation equipment according to claim 1, wherein: using a preset neural network model to respectively identify the obtained image data and environment perception data of the device to be detected comprises the following steps: for the device top image and the device middle image, a Cascade R-CNN algorithm is used as the backbone network, an RPN layer is added between the convolution layer and the pooling layer, background removal and device feature acquisition are performed, and device external state information is obtained according to the feature extraction results of the device top image and the device middle image.

3. The multi-dimensional image front-end analysis method suitable for oil leakage detection of power transformation equipment according to claim 2, wherein: using a preset neural network model to respectively identify the obtained image data and environment perception data of the device to be detected further comprises the following steps: ground state information is obtained by using an R-CNN neural network according to the obtained device ground image, and then a ground oil seepage identification result is obtained; device infrared abnormal areas are obtained by using an R-CNN network according to the obtained infrared detection image data, and whether the device leaks oil is determined by combining the device external state information and the distance between the infrared abnormal areas and the device oil storage points. comprise:

4. A multi-dimensional image front-end analysis system suitable for transformer oil leakage detection, characterized in that: a data acquisition module configured to obtain image data and environment perception data of a device to be detected; the image data of the device to be detected at least comprises a device top image, a device middle image and a device ground image, and the environment perception data at least comprises device infrared detection image data and device temperature detection data; an oil leakage identification module configured to use a preset neural network model to respectively identify the obtained image data and environment perception data of the device to be detected, and comprehensively analyze various identification results to obtain a final identification result of a device oil leakage defect. ​ Comprehensive various identification results, get the final identification result of the equipment oil leakage defect, including the following steps: If the overall equipment temperature is abnormal and there is oil seepage on the ground, it is determined that the equipment has oil leakage problem, and the specific oil leakage position is obtained by comprehensively judging the obtained infrared abnormal area of the equipment and the external state information of the equipment; Based on the multi-dimensional stereoscopic image, the oil carrying equipment of the transformer substation is analyzed, the surrounding multi-dimensional environment information is combined, it is judged whether the equipment leaks oil, and three cascaded R-CNN detectors are trained, each cascaded detector sets different IOU, and the output of the current network is used as the input of the next higher precision network.

5. A multi-dimensional image front-end analysis system suitable for oil leakage detection of power transformation equipment, characterized in that: It comprises: A plurality of arithmetic logic units, each of which is arranged in parallel, and different arithmetic logic units are configured to perform the calculation tasks of different network layers in the multi-dimensional image front-end analysis method suitable for oil leakage detection of power transformation equipment according to any one of claims 1-3; A temperature sensing acquisition unit for acquiring and judging the equipment temperature information and feeding back the abnormal information to the result output unit; A data processing unit configured to preprocess the collected data for oil leakage defects of power transformation equipment and send it to the corresponding arithmetic logic unit; A result output unit configured to integrate the operation data of each arithmetic logic unit and output the final operation result.

6. A multi-dimensional image front-end analysis system suitable for detecting oil leakage from power transformation equipment according to claim 5, characterized in that: The arithmetic logic unit comprises: A convolution activation layer operation unit configured to operate the convolution layer and the activation layer in the model and output the operation result; A pooling layer operation unit configured to operate the pooling layer in the model and output the operation result; A fully connected layer and output layer operation unit configured to operate the fully connected layer and the output layer in the model and output the operation result; A redundant operation unit configured to operate other network operation layers and output the operation result; Or, The operation tasks of each arithmetic logic unit are executed in parallel.

7. A robot, characterized by: A multi-dimensional image front-end analysis system suitable for oil leakage detection of power transformation equipment according to any one of claims 5 or 6.

8. A computer-readable storage medium, characterized in that: A plurality of instructions are stored therein, and the instructions are suitable for being loaded and executed by the processor of the terminal device to implement the steps of the multi-dimensional image front-end analysis method suitable for oil leakage detection of power transformation equipment according to any one of claims 1-3.

9. A terminal device, characterized by: It comprises a processor and a computer readable storage medium, the processor is used to realize each instruction; the computer readable storage medium is used to store a plurality of instructions, the instructions are suitable for being loaded and executed by the processor to implement the steps of the multi-dimensional image front-end analysis method suitable for oil leakage detection of power transformation equipment according to any one of claims 1-3.

Images