Electrical equipment surface defect image recognition and early warning system and related equipment

Abstract

The invention discloses a power equipment surface defect image recognition and early warning system and related equipment, which comprehensively utilizes the technical means of multi-modal data acquisition, deep learning model recognition, risk quantitative evaluation, trend prediction and the like by constructing a multi-module collaborative system architecture. And comprehensive detection and intelligent management of the surface defects of the power equipment are realized. Multi-modal sensing data are acquired through an image and data acquisition module, and the defect identification precision and the adaptability to complex defect characteristics are remarkably improved by combining an improved ResNet-50 network and a defect identification and positioning module of a YOLOv5 target detection algorithm. And the defect risk assessment and trend prediction module adopts defect area ratio calculation and a long short-term memory (LSTM) network, so that quantitative analysis of defect risks and accurate prediction of an expansion trend are realized, and a reliable basis is provided for operation state assessment of power equipment.

Description

Technical Field

[0001] The present invention belongs to the technical field of power equipment detection and intelligent operation and maintenance, and particularly relates to a power equipment surface defect image recognition and early warning system and related equipment. Background Art

[0002] In modern power systems, the stable operation of power equipment plays a crucial role in the safety and reliability of the entire power grid. During the long-term operation of power equipment, under the combined influence of external environment, mechanical stress, thermal cycle and electrical stress, various defects are likely to occur on its surface, such as cracks, corrosion, rust and stains. If these defects are not detected and processed in time, they may cause equipment failures and even lead to serious accidents in the power system. Therefore, researching the detection and management technologies for power equipment surface defects is of great significance for improving the operation reliability of equipment and extending the service life of equipment.

[0003] Currently, the detection technologies for power equipment surface defects mainly include manual detection, traditional image processing and data analysis based on single-modal sensors. Manual detection relies on the experience and subjective judgment of inspection personnel, with low detection efficiency and being easily interfered by external conditions; traditional image processing technologies achieve automatic defect recognition through algorithms such as edge detection and gray-scale analysis, but have poor adaptability to complex defects; data analysis based on single-modal sensors such as infrared thermal imaging or ultrasonic detection can provide detection results with a certain degree of accuracy in specific scenarios, but have limited comprehensive detection capabilities for various defect types. There is still much room for improvement in the comprehensiveness, real-time performance and intelligent level of these technical means for defect detection. Summary of the Invention

[0004] The purpose of the present invention is to provide a power equipment surface defect image recognition and early warning system and related equipment to solve the technical problems of poor comprehensiveness and accuracy in the detection of power equipment surface defects in the prior art.

[0005] To achieve the above object, the present invention adopts the following technical solutions: A method for power equipment surface defect image recognition and early warning, comprising the following steps: Collect multi-modal data on the surface of the equipment to obtain visible light images, temperature distribution data, ultrasonic reflection data and three-dimensional point cloud models of the equipment; Perform denoising, light correction, edge enhancement and data fusion processing on the collected data; For the preprocessed data, feature extraction is performed through an improved ResNet-50 network combined with an attention mechanism, and defect recognition and positioning are carried out through the YOLOv5 target detection algorithm, and the type, position coordinates, bounding box size and confidence score of the defect are output; According to the type and location information of the defect, calculate the risk level of the defect through the defect area ratio calculation and weighted risk scoring model, and predict the defect expansion trend based on the long short-term memory network; Receive the recognition result, risk assessment data and trend prediction data, store them through blockchain technology, and generate warning feedback information.

[0006] Furthermore, cloud-edge collaboration technology is also adopted to perform real-time defect analysis at the edge, perform in-depth analysis and model optimization in the cloud, and send the optimized model to the edge device.

[0007] Furthermore, the calculation formula for the defect area ratio is:

[0008] In the formula, represents the defect area, represents the total surface area of the device.

[0009] Furthermore, the improved ResNet-50 network combines an attention mechanism to perform weighted processing on key defect areas, highlighting crack, rust, and corrosion feature areas.

[0010] In a second aspect, the present invention provides an image recognition and early warning system for surface defects of power equipment, including an image and data acquisition module, an image preprocessing module, a defect recognition and location module, a defect risk assessment and trend prediction module, and a blockchain management module connected in sequence, where: Image and data acquisition module: used to collect multi-modal data on the surface of the device, and obtain visible light images, temperature distribution data, ultrasonic reflection data, and three-dimensional point cloud models of the device; Image preprocessing module: used to perform denoising, illumination correction, edge enhancement, and data fusion processing on the collected data; Defect recognition and location module: used to extract features from the preprocessed data through the improved ResNet-50 network combined with an attention mechanism, and identify and locate the defect through the YOLOv5 object detection algorithm, and output the type, position coordinates, bounding box size, and confidence score of the defect; Defect risk assessment and trend prediction module: used to calculate the risk level of the defect through the defect area ratio calculation and weighted risk scoring model according to the type and location information of the defect, and predict the defect expansion trend based on the long short-term memory network; Blockchain management module: used to receive the recognition result, risk assessment data, and trend prediction data, store them through blockchain technology, and generate warning feedback information.

[0011] Further, the defect risk assessment and trend prediction module is also connected to an autonomous inspection path planning module, and the image and data acquisition module is bidirectionally connected to the autonomous inspection path planning module.

[0012] Further, the autonomous inspection path planning module is used to dynamically plan the inspection path of the drone or ground robot through a reinforcement learning algorithm, and optimize the inspection strategy according to the recognition result.

[0013] Further, a cloud-edge collaborative computing module is also provided between the defect risk assessment and trend prediction module and the blockchain management module. The cloud-edge collaborative computing module is used to perform real-time defect analysis at the edge, perform in-depth analysis and model optimization in the cloud, and send the optimized model to the edge device.

[0014] In a third aspect, a terminal device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the above method are implemented.

[0015] In a fourth aspect, a computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0016] Compared with the prior art, the present invention has the following beneficial technical effects: A method for identifying and warning surface defects of power equipment according to the present invention uses multi-modal data acquisition, combines defect recognition and positioning of an improved ResNet-50 network and YOLOv5 object detection algorithm, and significantly improves the accuracy of defect recognition and the adaptability to complex defect features; the defect area ratio calculation and long short-term memory network are used to realize the quantitative analysis of defect risks and the accurate prediction of expansion trends, providing a reliable basis for the operation status evaluation of power equipment; the defect recognition results, risk assessment and trend prediction data are stored through blockchain technology to ensure the immutability and traceability of data. The operation and maintenance tasks are managed through smart contracts to improve the data security and transparency of the system. The present invention realizes the comprehensive detection and intelligent warning of surface defects of power equipment through multi-modal data acquisition, deep learning defect recognition, quantitative risk assessment and trend prediction, and blockchain data management, overcomes the deficiencies in the prior art, and has significant innovation and practical value.

[0017] Preferably, the present invention also adopts cloud-edge collaborative technology to perform real-time defect analysis at the edge, perform in-depth analysis and model optimization in the cloud, and send the optimized model to the edge device, realizing real-time recognition of the edge device, in-depth analysis and model update in the cloud, and forming a data closed-loop processing.

[0018] The present invention also provides a power equipment surface defect image recognition and early warning system. By comprehensively utilizing technical means such as multi-modal data acquisition, deep learning model recognition, risk quantification assessment, and trend prediction, the comprehensiveness, intelligence, and data security of power equipment defect detection are overall improved, and comprehensive detection and intelligent management of power equipment surface defects are realized.

[0019] Preferably, the system further includes an autonomous inspection path planning module. By optimizing the inspection path through a reinforcement learning algorithm, the efficiency and coverage rate of equipment inspection are significantly improved.

[0020] Preferably, the system further includes a cloud-edge collaborative computing module. By combining the cloud-edge collaborative computing module with the blockchain management module, the real-time nature of data processing and the security of storage are ensured. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a flowchart of a method for power equipment surface defect image recognition and early warning in an embodiment of the present invention; Figure 2 It is a schematic diagram of the modules of a power equipment surface defect image recognition and early warning system in an embodiment of the present invention; Figure 3 It is a flowchart of a method for power equipment surface defect image recognition and early warning in another embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0022] In order to enable those skilled in the art to better understand the solution of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0023] It should be noted that: similar reference numerals and letters denote similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0024] It should be noted that the terms "first", "second", etc. in the description, claims and above-mentioned drawings of the present invention are used to distinguish similar objects, and do not necessarily need to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device comprising a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0025] The following further describes the present invention in detail with reference to the drawings: As Figure 1 shown, a method for identifying and warning surface defects of power equipment includes the following steps: Step 1, collect multi-modal data on the surface of the equipment to obtain visible light images, temperature distribution data, ultrasonic reflection data and three-dimensional point cloud models of the equipment; Among them, high-definition visible light cameras, infrared thermal imagers, ultrasonic imaging devices and lidars are used to collect surface data of the equipment, and drones, ground robots and fixed monitoring devices are used for flexible deployment and obtaining multi-angle data.

[0026] Step 2, perform denoising, illumination correction, edge enhancement and data fusion processing on the collected data; Step 3, for the preprocessed data, perform feature extraction through an improved ResNet-50 network combined with an attention mechanism, and identify and locate defects through the YOLOv5 object detection algorithm, and output the type, position coordinates, bounding box size and confidence score of the defects; The improved ResNet-50 network combined with the attention mechanism is used to perform weighted processing on key defect areas, highlight crack, rust and corrosion feature areas, and classify, identify and locate defects by using the improved ResNet50 network and the YOLOv5 object detection algorithm.

[0027] Step 4, according to the type and location information of the defects, evaluate the risk level of the defects through defect area ratio calculation and weighted risk scoring model, and predict the defect expansion trend based on the long short-term memory network; Evaluate the defect level through the defect area ratio formula and the risk scoring model. The defect area ratio calculation formula is:

[0028] In the formula, represents the defect area, Represents the total surface area of the device; The risk scoring model is calculated using the following formula:

[0029] Wherein, is the risk score, is the defect area ratio, is the defect type weight, is the device operating environment weight, , , are risk assessment weight factors, satisfying + + = 1.

[0030] Based on the LSTM model, predict the defect expansion trend and generate future risk level change information.

[0031] Step five, receive the recognition result, risk assessment data and trend prediction data, store them through blockchain technology, and generate early warning feedback information; Store the recognition result and evaluation information on the chain to generate early warning feedback information.

[0032] In another embodiment of the present invention, a method for identifying and warning surface defects of power equipment is provided. As Figure 3 shown, cloud-edge collaboration technology is also adopted. Real-time defect analysis is carried out at the edge, in-depth analysis and model optimization are carried out in the cloud, and the optimized model is sent to the edge device, that is, the edge device performs real-time recognition, the cloud performs in-depth analysis and model update, forming a closed-loop data processing.

[0033] A power equipment surface defect image recognition and warning system includes an image and data acquisition module, an image preprocessing module, a defect recognition and positioning module, a defect risk assessment and trend prediction module, and a blockchain management module connected in sequence, wherein: Image and data acquisition module: used to collect multi-modal data on the surface of the device, and obtain visible light images, temperature distribution data, ultrasonic reflection data and 3D point cloud models of the device; Image preprocessing module: used to perform denoising, illumination correction, edge enhancement and data fusion processing on the collected data, including a Gaussian filtering unit, a histogram equalization unit and an edge detection unit, which are used to realize image denoising, illumination correction and edge enhancement respectively; Defect Identification and Location Module: For the preprocessed data, it extracts features through an improved ResNet-50 network combined with an attention mechanism, and identifies and locates defects through the YOLOv5 object detection algorithm, outputting the type, position coordinates, bounding box size, and confidence score of the defects; Defect Risk Assessment and Trend Prediction Module: For the type and location information of the defects, it evaluates the risk level of the defects through defect area ratio calculation and a weighted risk scoring model, and predicts the defect expansion trend based on a long short-term memory network; Blockchain Management Module: For receiving the recognition results, risk assessment data, and trend prediction data, it stores them through blockchain technology and generates warning feedback information.

[0034] In another embodiment of the present invention, a power equipment surface defect image recognition and warning system is provided, as Figure 2 shown, including the following modules: Image and Data Acquisition Module, Image Preprocessing Module, Defect Identification and Location Module, Defect Risk Assessment and Trend Prediction Module, Autonomous Inspection Path Planning Module, Cloud-Edge Collaborative Computing Module, and Blockchain Management Module.

[0035] Figure 2 Shows the interaction relationships and data flows between the modules, reflecting the overall operation logic of the system and the collaborative effects of the modules. Through the input provided by the Image and Data Acquisition Module, the Image Preprocessing Module optimizes the data, the Defect Identification and Location Module completes classification and location, the Risk Assessment and Trend Prediction Module performs risk quantification and trend prediction, the Autonomous Inspection Path Planning Module dynamically adjusts the inspection plan, the Cloud-Edge Collaborative Computing Module realizes the data processing closed-loop, and finally the Blockchain Management Module realizes the secure storage and feedback warning of the data. Through the efficient interaction between the modules, this system realizes the accurate identification and intelligent warning of power equipment defects; specifically as follows: 1. Image and Data Acquisition Module This module realizes the comprehensive acquisition of multi-modal data on the surface of power equipment, combined with the following sensing devices and acquisition strategies; 1) High-definition visible light camera: Captures high-definition images of the equipment surface for identifying external defects such as cracks and rust.

[0036] 2) Infrared thermal imager: Obtains the temperature distribution map of the equipment surface for detecting defects caused by overheating or abnormal heat diffusion.

[0037] 3) Ultrasonic imaging device: Detects surface micro-cracks and internal defects, and forms defect images through reflected waves.

[0038] 4) LiDAR scanning device: Generates three-dimensional point cloud data of the power equipment surface, reflecting the overall spatial characteristics of the equipment.

[0039] The acquisition device is flexibly deployed through drones, ground robots and fixed monitoring devices. Drones are used for the inspection of transmission lines, ground robots are suitable for the complex inspection of internal equipment in substations, and fixed devices are used for the routine monitoring of equipment. Multi-sensor collaborative work realizes time synchronization and spatial alignment to ensure the comprehensiveness of the collected data.

[0040] 2. Image preprocessing module The collected multi-modal data is processed by the image preprocessing module for denoising, illumination correction, edge enhancement and data fusion.

[0041] Denoising: Gaussian filtering is used to remove image noise and improve image clarity.

[0042] Illumination correction: Histogram equalization is used to process images with uneven illumination to improve the consistency of gray-scale distribution.

[0043] Edge enhancement: Canny operator is used to extract the edge features of cracks and rust.

[0044] Data fusion: The weighted average fusion algorithm is used to perform spatio-temporal alignment on infrared, ultrasonic and visible light images to generate a unified high-quality image.

[0045] 3. Defect recognition and localization module This module automatically recognizes and locates surface defects of equipment through a deep learning model.

[0046] Feature extraction: An improved ResNet50 network is used to extract multi-scale features, and an attention mechanism (SE module) is introduced to weight key defect areas.

[0047] Object detection: The YOLOv5 object detection algorithm is used to classify and locate defects, and output the defect type, position coordinates , bounding box size and confidence score.

[0048] The system output includes the classification results and accurate positions of defects such as cracks, rust, and corrosion.

[0049] 4. Defect risk assessment and trend prediction module The identified defects are quantified and predicted by the risk assessment and trend prediction module.

[0050] Calculation of defect area ratio: Quantify the severity of the defect:

[0051] Among them, is the defect area, is the total surface area of the equipment.

[0052] Risk scoring model: Combine the defect type and environment to calculate the risk level:

[0053] Among them, is the risk score, is the defect area ratio, is the defect type weight, is the equipment operating environment weight, , , are risk assessment weight factors, satisfying + + = 1.

[0054] Trend prediction: Use a long short-term memory network (LSTM) to model the defect expansion:

[0055] Among them, is the current predicted value, is the historical defect data, is the network parameter.

[0056] The system output includes the future expansion area of the defect, the expansion speed, and the change in the risk level.

[0057] 5. Autonomous inspection path planning module This module uses a reinforcement learning algorithm (Deep QLearning) to plan the inspection path for drones or ground robots, constructs a 3D environment model around the equipment through the 3D point cloud data generated by lidar, and realizes dynamic planning of the optimal inspection path.

[0058] 6. Cloud-edge collaborative computing module Edge devices are responsible for real-time defect detection, and the cloud performs in-depth analysis and model optimization, and then sends the optimized model to the edge devices to achieve collaborative work of fast recognition at the edge and precise analysis in the cloud.

[0059] 7. Blockchain management module Store the defect recognition results, risk assessment, and trend prediction data through blockchain technology to ensure the immutability and traceability of the data. Manage the operation and maintenance tasks through smart contracts to improve the data security and transparency of the system.

[0060] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0061] The present invention is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to embodiments of the present invention. It should be understood that each flow and / or block in the flowchart and / or block diagram, as well as the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0062] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including instruction means that implement the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0063] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0064] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit its protection scope. Although the present invention has been described in detail with reference to the above embodiments, those of ordinary skill in the art should understand that after reading the present invention, various changes, modifications, or equivalent replacements can still be made to the specific implementation manners of the invention, but these changes, modifications, or equivalent replacements are all within the protection scope of the pending claims of the invention.

Claims

1. A method for image recognition and early warning of surface defects of power equipment, characterized in that: The following steps are involved: Collect multimodal data on the surface of the equipment, obtain visible light images, temperature distribution data, ultrasonic reflection data and three-dimensional point cloud models of the equipment; Perform denoising, illumination correction, edge enhancement and data fusion processing on the collected data; For the preprocessed data, the improved ResNet-50 network combined with the attention mechanism is used to extract features, and the defects are identified and located using the YOLOv5 target detection algorithm, outputting the defect type, location coordinates, bounding box size, and confidence score; According to the defect type and location information, the risk level of the defect is evaluated through defect area ratio calculation and weighted risk scoring model, and the defect expansion trend is predicted based on long short-term memory network; Receive identification results, risk assessment data and trend forecast data, store them through blockchain technology, and generate early warning feedback information.

2. A method for surface defect image recognition and early warning of power equipment according to claim 1, characterized in that: It also uses cloud-edge collaborative technology to perform real-time defect analysis at the edge, deep analysis and model optimization in the cloud, and send the optimized model to edge devices.

3. The method for surface defect image recognition and early warning of electric power equipment according to claim 1, characterized in that: The defect area ratio calculation formula is: In the formula, represents the defect area, Represents the total surface area of ​​the equipment.

4. The method for surface defect image recognition and early warning of electric power equipment according to claim 1, characterized in that: The improved ResNet-50 network is combined with the attention mechanism to perform weighted processing on key defect areas and highlight the characteristic areas of cracks, rust and corrosion.

5. An image recognition and early warning system for surface defects of power equipment, characterized in that: The method for surface defect image recognition and early warning of power equipment according to any one of claims 1 to 4 comprises an image and data acquisition module, an image preprocessing module, a defect recognition and positioning module, a defect risk assessment and trend prediction module and a blockchain management module connected in sequence, wherein: Image and data acquisition module: used to collect multimodal data on the surface of the equipment, obtain the visible light image, temperature distribution data, ultrasonic reflection data and three-dimensional point cloud model of the equipment; Image preprocessing module: used to perform denoising, illumination correction, edge enhancement and data fusion processing on the collected data; Defect recognition and positioning module: It is used to extract features from preprocessed data through the improved ResNet-50 network combined with the attention mechanism, and to identify and locate defects through the YOLOv5 target detection algorithm, and output the defect type, location coordinates, bounding box size and confidence score; Defect risk assessment and trend prediction module: used to assess the risk level of defects based on the defect type and location information through defect area ratio calculation and weighted risk scoring model, and predict the defect expansion trend based on long short-term memory network; Blockchain management module: used to receive identification results, risk assessment data and trend forecast data, store them through blockchain technology, and generate early warning feedback information.

6. The surface defect image recognition and early warning system for electric power equipment according to claim 5, characterized in that: The defect risk assessment and trend prediction module is also connected to the autonomous inspection path planning module, and the image and data acquisition module is bidirectionally connected to the autonomous inspection path planning module.

7. The surface defect image recognition and early warning system for electric power equipment according to claim 6, characterized in that: The autonomous inspection path planning module is used to dynamically plan the inspection path of the drone or ground robot through a reinforcement learning algorithm, and optimize the inspection strategy based on the recognition results.

8. The surface defect image recognition and early warning system for electric power equipment according to claim 5, characterized in that: A cloud-edge collaborative computing module is also provided between the defect risk assessment and trend prediction module and the blockchain management module. The cloud-edge collaborative computing module is used to perform real-time defect analysis at the edge, perform in-depth analysis and model optimization in the cloud, and send the optimized model to the edge device.

9. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 4 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images