Intelligent maintenance system and method based on image analysis

Abstract

The invention discloses an intelligent maintenance system and method based on image analysis, and the system comprises a data collection module, a data fusion module, an intelligent analysis module, and a decision alarm module, the data collection module is used for collecting multi-dimensional image data and equipment data of a target power transformation main device in a target power system, obtaining first multi-dimensional image data and target equipment data; the data fusion module is used for fusing images in the first multi-dimensional image data to obtain a target fusion image; the intelligent analysis module is used for performing fault identification on the target power transformation main equipment according to the target fusion image and the target equipment data to obtain a target fault identification result; and the decision alarm module is used for determining a target maintenance plan and a target alarm operation corresponding to the target fault recognition result. According to the invention, the fault diagnosis accuracy of the power transformation main equipment can be improved.

Description

Technical Field

[0001] This application relates to the technical field of fault detection, and particularly to an intelligent maintenance system and method based on image analysis. Background Art

[0002] In the power system, as the key hub for power transmission and distribution, the stable operation of the main substation equipment is crucial for ensuring the reliability and security of power supply.

[0003] Traditional fault maintenance often can only obtain single-dimensional information of the equipment, such as appearance, temperature, etc., and it is difficult to comprehensively and deeply understand the operating state of the equipment. Therefore, the accuracy of fault diagnosis is relatively low. Thus, how to improve the accuracy of fault diagnosis of the main substation equipment has become an urgent problem to be solved. Summary of the Invention

[0004] The embodiments of this application provide an intelligent maintenance system and method based on image analysis, which can improve the accuracy of fault diagnosis of the main substation equipment.

[0005] In a first aspect, the embodiments of this application provide an intelligent maintenance system based on image analysis. The system includes: a data acquisition module, a data fusion module, an intelligent analysis module, and a decision-making and alarm module, where:

[0006] The data acquisition module is used to collect multi-dimensional image data and equipment data of the target main substation equipment in the target power system to obtain first multi-dimensional image data and target equipment data;

[0007] The data fusion module is used to fuse the images in the first multi-dimensional image data to obtain a target fused image; the target fused image is a three-dimensional stereoscopic image;

[0008] The intelligent analysis module is used to perform fault identification on the target main substation equipment according to the target fused image and the target equipment data to obtain a target fault identification result;

[0009] The decision-making and alarm module is used to determine a target maintenance plan and a target alarm operation corresponding to the target fault identification result.

[0010] In a second aspect, the embodiments of this application provide an intelligent maintenance method based on image analysis. The method includes:

[0011] Collect multi-dimensional image data and equipment data of the target main substation equipment in the power system to obtain first multi-dimensional image data and target equipment data;

[0012] Fuse the images in the first multi-dimensional image data to obtain a target fused image; the target fused image is a three-dimensional stereoscopic image;

[0013] Perform fault identification on the target main power transformation equipment according to the target fusion image and the target device data to obtain a target fault identification result;

[0014] Determine a target maintenance plan and a target alarm operation corresponding to the target fault identification result.

[0015] In a third aspect, an embodiment of the present application provides an electronic device, including: a processor, a memory, a communication interface, and one or more programs, wherein the above one or more programs are stored in the above memory and are configured to be executed by the above processor, and the above programs include instructions for performing the steps in the second aspect of the embodiment of the present application.

[0016] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the above computer-readable storage medium stores a computer program for electronic data exchange, and wherein the above computer program enables a computer to execute some or all of the steps described in the second aspect of the embodiment of the present application.

[0017] In a fifth aspect, an embodiment of the present application provides a computer program product, wherein the above computer program product includes a non-transitory computer-readable storage medium storing a computer program, and the above computer program is operable to enable a computer to execute some or all of the steps described in the second aspect of the embodiment of the present application. The computer program product can be a software installation package.

[0018] Implementing the present application has the following beneficial effects:

[0019] It can be seen that the intelligent maintenance system based on image analysis described in the present application includes: a data acquisition module, a data fusion module, an intelligent analysis module, and a decision-making alarm module, wherein: the data acquisition module is used to acquire multi-dimensional image data and device data of the target main power transformation equipment in the target power system to obtain first multi-dimensional image data and target device data; the data fusion module is used to fuse the images in the first multi-dimensional image data to obtain a target fusion image; the target fusion image is a three-dimensional stereoscopic image; the intelligent analysis module is used to perform fault identification on the target main power transformation equipment according to the target fusion image and the target device data to obtain a target fault identification result; the decision-making alarm module is used to determine a target maintenance plan and a target alarm operation corresponding to the target fault identification result; thus, fault identification is performed based on the target fusion image and the target device data, and by integrating these two pieces of information, it is possible to more accurately judge whether the fault is an external fault or an internal fault of the device, and moreover, the two pieces of information can also mutually verify and confirm the fault, thereby more accurately identifying the fault type, location, and severity, and further improving the accuracy of fault diagnosis of the main power transformation equipment. Description of the Drawings

[0020] To more clearly illustrate the technical solutions in the embodiments of the present application or the background art, the following will describe the drawings required for use in the embodiments of the present application or the background art.

[0021] Figure 1 It is an application scenario diagram of an intelligent maintenance system based on image analysis provided by an embodiment of the present application;

[0022] Figure 2 It is a schematic structural diagram of an intelligent maintenance system based on image analysis provided by an embodiment of the present application;

[0023] Figure 3 It is a schematic structural diagram of a data fusion module provided by an embodiment of the present application;

[0024] Figure 4 It is a schematic structural diagram of an intelligent analysis module provided by an embodiment of the present application;

[0025] Figure 5 It is a schematic structural diagram of a decision-making and alarm module provided by an embodiment of the present application;

[0026] Figure 6 It is a schematic structural diagram of a visualization display module provided by an embodiment of the present application;

[0027] Figure 7 It is a flowchart of an intelligent maintenance method based on image analysis provided by an embodiment of the present application;

[0028] Figure 8 It is a schematic structural diagram of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0029] In order to enable those skilled in the art to better understand the solution of the present application, the following will clearly and completely describe the technical solutions in the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without making creative efforts belong to the scope of protection of the present application.

[0030] In the description and claims of this application and the above-mentioned drawings, terms such as "first" and "second" are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally further include steps or units not listed, or may optionally further include other steps or units inherent to these processes, methods, products or devices.

[0031] It should be understood that the term "and / or" in this article is merely a description of the association relationship between associated objects, indicating that there can be three relationships. For example, A and / or B can represent the following three situations: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " in this article indicates that the associated objects before and after are in an "or" relationship. The "plurality" mentioned in the embodiments of this application refers to two or more.

[0032] The "at least one (piece)" or its similar expression in the embodiments of this application refers to any combination of these items, including any combination of single items (pieces) or plural items (pieces), and refers to one or more, and plural refers to two or more. For example, at least one (piece) of a, b, or c can represent the following seven situations: a, b, c, a and b, a and c, b and c, a, b, and c. Among them, each of a, b, and c can be an element or a set containing one or more elements.

[0033] The "connection" that appears in the embodiments of this application refers to various connection methods such as direct connection or indirect connection to achieve communication between devices, and this application does not make any limitations on this.

[0034] Referring to "embodiments" in this article means that the specific features, structures, or characteristics described in connection with the embodiments may be included in at least one embodiment of this application. The appearance of this phrase in various positions in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. Those skilled in the art will explicitly and implicitly understand that the embodiments described herein can be combined with other embodiments.

[0035] The electronic devices described in the embodiments of this application may include smartphones (such as Android phones, iOS phones, Windows Phone phones, etc.), tablet computers, handheld computers, laptop computers, video matrices, monitoring platforms, mobile internet devices (MID), or wearable devices, etc. The above are only examples and not exhaustive, including but not limited to the above-mentioned devices.

[0036] Of course, the above electronic device can also be a server, for example, a cloud server.

[0037] The relevant content, concepts, meanings, technical problems, technical solutions, beneficial effects, etc. involved in the embodiments of the present application will be described below.

[0038] First, some professional terms involved in the present application will be explained:

[0039] The main substation equipment in the power system: refers to the key equipment that undertakes the main functions of electric energy conversion, distribution and control in the substation link of the power system. Common main substation equipment includes transformers (realize voltage level conversion and reduce power transmission losses), circuit breakers (can connect or disconnect circuits under normal and fault conditions), disconnecting switches (used to isolate the power supply and ensure maintenance safety), instrument transformers (including voltage transformers and current transformers, used for measurement and protection), etc. They are the core components of substations, and their reliable operation is crucial for the stability of the power system.

[0040] Scale-Invariant Feature Transform (SIFT) algorithm: It is a feature extraction algorithm in the field of computer vision. It can detect feature points with scale invariance in images, that is, no matter whether the image is enlarged or reduced, these feature points can exist stably and be recognized. By calculating information such as the gradient direction and amplitude in the area around the feature points, unique feature descriptors are generated for tasks such as image matching, object recognition, and image stitching. For example, when photographing the same substation equipment at different shooting distances, the SIFT algorithm can extract the same key features for subsequent analysis.

[0041] Fast Library for Approximate Nearest Neighbors (FLANN): It is an open-source library for quickly finding the nearest neighbor points in a dataset. When dealing with high-dimensional data (such as a large number of image feature vectors), the traditional nearest neighbor search method has a huge computational amount and is time-consuming. FLANN provides a series of optimized algorithms and data structures, which can significantly improve the search efficiency in approximate nearest neighbor search and is commonly used in fields such as image recognition and clustering analysis. For example, in the image feature matching of substation equipment, it can quickly find similar feature images.

[0042] Perspective transformation method: A geometric transformation method that projects an image onto a new viewing plane. Based on the principle of central projection, it can simulate perspective effects such as objects appearing larger when closer and smaller when farther away, and parallel lines converging at vanishing points when observed by the human eye. In image processing, it can be used to correct image distortion caused by shooting angle problems, or to convert a 2D image into a 3D effect with a specific viewing angle, making the image more conform to human visual habits, and has applications in aspects such as adjusting the shooting angle and 3D reconstruction of substation equipment images.

[0043] Preset multi-view stereo vision algorithm: An algorithm that reconstructs a 3D model of an object or obtains the depth information of an object based on images taken from multiple different viewpoints. By analyzing the corresponding relationships of feature points of the object in images from different viewpoints and using principles such as triangulation to calculate the spatial coordinates of each point of the object, a 3D model is then constructed. In substation equipment detection, images of the equipment can be taken from different angles, and then this algorithm can be used to reconstruct the 3D model of the equipment to more comprehensively detect equipment appearance defects, spatial position relationships, etc.

[0044] Long short-term memory network model: A special type of recurrent neural network (RNN). Traditional RNNs have problems of vanishing gradients or exploding gradients when dealing with long sequence data, making it difficult to learn long-distance dependencies. LSTM can selectively remember and forget information by introducing memory cells and gating mechanisms (input gate, forget gate, output gate), and can effectively process and preserve long-term dependencies in long sequence data, and is widely used in fields such as time series data processing (such as power load forecasting, time series analysis of equipment status monitoring), speech recognition, and natural language processing.

[0045] Knowledge graph: A knowledge representation form based on semantic networks, which graphically shows entities (such as substation equipment, fault types, maintenance tools, etc.) and the relationships between them (such as the association between equipment and faults, the association between faults and maintenance methods, etc.). It structures a large amount of domain knowledge, with nodes representing entities and edges representing relationships between entities, and can quickly query and reason about knowledge, providing knowledge support for intelligent question answering, fault diagnosis reasoning, intelligent decision-making, etc., and can be used to store and retrieve equipment-related knowledge in intelligent maintenance of substation equipment.

[0046] Fault diagnosis logical reasoning chain: In the process of fault diagnosis, it is a set of a series of reasoning steps that, based on certain logical rules, start from the fault phenomenon and gradually deduce the fault cause. It is based on expert knowledge, equipment operation principles, historical fault cases, etc., to establish a logical mapping relationship between the fault phenomenon and the fault cause. For example, when the temperature of a substation equipment rises abnormally, through the fault diagnosis logical reasoning chain, possible causes such as heat dissipation system failure, overloading, and poor internal contact are considered in turn, and the real fault cause is gradually identified through investigation.

[0047] Please refer toFigure 1 , Figure 1 is an application scenario diagram of an intelligent maintenance system based on image analysis provided by an embodiment of the present application. It can be seen that the intelligent maintenance system based on image analysis can be physically or communicatively connected to the target main substation equipment in the target power system to collect multi-dimensional image data and equipment data of the target main substation equipment. When it is necessary to identify faults in the target main substation equipment, the intelligent maintenance system based on image analysis can analyze based on the multi-dimensional image data and equipment data, thereby obtaining an identification result. Finally, corresponding maintenance plans and alarm operations can also be formulated according to the identification result.

[0048] Please refer to Figure 2 , Figure 2 is a schematic structural diagram of an intelligent maintenance system based on image analysis provided by an embodiment of the present application. It can be seen that the intelligent maintenance system based on image analysis includes: a data acquisition module, a data fusion module, an intelligent analysis module, and a decision-making alarm module, where:

[0049] The data acquisition module is used to collect multi-dimensional image data and equipment data of the target main substation equipment in the target power system to obtain first multi-dimensional image data and target equipment data;

[0050] In an embodiment of the present application, the target main substation equipment may be at least one of the following: transformer, switchgear, instrument transformer, lightning arrester, etc., which is not limited herein; the target equipment data may include at least one of the following: voltage data, current data, temperature data, etc., which is not limited herein.

[0051] In a specific embodiment, the data acquisition module may include a three-dimensional space acquisition unit (for example, lidar), a visible light acquisition unit (for example, a camera), a thermal imaging acquisition unit (for example, an infrared thermal imager), and a substation equipment status monitoring unit (for example, a monitoring sensor group). The lidar, high-definition camera, and infrared thermal imager can be deployed at appropriate positions in the target power system to ensure that the target main substation equipment can be fully covered and data acquisition dead angles can be avoided. Then, the parameters of the lidar, high-definition camera, and infrared thermal imager can be set according to the size, distance, and acquisition requirements of the target main substation equipment. For example, the scanning angle, resolution, and scanning frequency of the lidar can be set; the focal length, aperture, and shooting frame rate of the high-definition camera can be set; the temperature measurement range and temperature resolution of the infrared thermal imager can be set.

[0052] Further, when it is necessary to collect data of the target main substation equipment, the system can send a synchronous acquisition signal to the data acquisition module to enable the lidar, high-definition camera, and infrared thermal imager to start collecting data simultaneously. This can ensure that the visible light image, thermal imaging image, and three-dimensional space image obtained are the states of the main substation equipment at the same moment, facilitating subsequent data fusion and analysis. Specifically, the lidar can obtain the three-dimensional space coordinate information of the target main substation equipment by emitting laser beams and measuring the time of the reflected light, and generate a three-dimensional space image. The high-definition camera uses an optical lens to form an image, and converts the optical signal into an electrical signal through an image sensor to obtain the appearance detail image of the target main substation equipment, that is, the visible light image. The infrared thermal imager detects the infrared radiation energy emitted from the surface of the main substation equipment, converts it into temperature information, and presents it in the form of a thermal image to obtain a thermal imaging image.

[0053] Next, the data acquisition module can also verify the visible light image, thermal imaging image, and three-dimensional space image data collected, checking the integrity, accuracy, and consistency of the data, such as whether there are problems such as missing images, data errors, and inconsistent timestamps. For the data that fails the verification, it is marked or recollected to ensure that the quality of the collected data meets the requirements. If all the data passes the verification, the first multi-dimensional image data is composed of the visible light image, thermal imaging image, and three-dimensional space image data.

[0054] Then, the target main substation equipment can be monitored in real time through the substation equipment status monitoring unit to obtain the target equipment data. Specifically, the operating parameters of the target main substation equipment to be monitored can be determined first. For example, the oil temperature, oil level, and winding temperature of the transformer, the contact temperature, opening and closing position, and operating mechanism pressure of the circuit breaker, the current, voltage, and temperature of the bus, etc. Then, according to different operating parameters, appropriate sensors are selected for monitoring to obtain the target equipment data. For example, thermocouples or thermal resistance sensors can be used to measure the equipment temperature, capacitive or float sensors can be used to measure the oil level, and current transformers and voltage transformers can be used to measure the current and voltage, etc. This is not limited here.

[0055] It should be noted that the intelligent maintenance system based on image analysis can also include: a data storage module, etc., which is not limited here. The data storage module is used to save the multi-dimensional image data and equipment data of the target main substation equipment obtained by the data acquisition module, such as equipment appearance images at different angles, thermal imaging images, equipment operating parameters (voltage, current, etc.), historical maintenance records, etc. These original data are the basis for subsequent analysis and fault judgment, and are completely retained for easy backtracking and further analysis at any time.

[0056] The data fusion module is used to fuse the images in the first multi-dimensional image data to obtain a target fusion image; the target fusion image is a three-dimensional stereoscopic image;

[0057] In the embodiments of the present application, the images in the first multi-dimensional image data can be pre-processed (e.g., denoised) first, and then the images in the first multi-dimensional image data can be fused to obtain a target fused image.

[0058] Optionally, the first multi-dimensional image data includes: visible light images, thermal imaging images, three-dimensional space images; for fusing the images in the first multi-dimensional image data to obtain a target fused image, please refer to Figure 3 , Figure 3 is a schematic structural diagram of a data fusion module provided by the embodiments of the present application. As Figure 3 shown, the data fusion module includes: a feature point matching unit, a spatio-temporal registration unit, a three-dimensional modeling and stereoscopic unit, and a multi-source data fusion unit, where:

[0059] The feature point matching unit is configured to process the first multi-dimensional image data according to the SIFT algorithm to obtain three feature point sets and three feature vector sets; each dimension data (or each image) corresponds to a feature point set and a feature vector set; the fast nearest neighbor search library is used to match the feature points in the three feature point sets to obtain a target matching relationship.

[0060] In the embodiments of the present application, both the preset perspective transformation method and the preset multi-view stereo vision algorithm can be preset in advance or by default.

[0061] In specific embodiments, the SIFT algorithm can be used to process the visible light images, thermal imaging images, and three-dimensional space images in the first multi-dimensional image data respectively to obtain three feature point sets (respectively corresponding to the feature point sets of the three types of images) and three feature vector sets (respectively corresponding to the feature vector sets of the three types of images). Since the SIFT algorithm is a conventional technology, it will not be elaborated here.

[0062] Next, a fast nearest neighbor search library can be used to match the feature points in the three feature point sets to obtain the target matching relationship. Specifically, the fast nearest neighbor search library can be the FLANN (Fast Library for Approximate Nearest Neighbors) search library. Functions provided by the fast nearest neighbor search library can be used to construct an index structure for each feature point set based on the feature vector set. For example, in the FLANN search library, an efficient index can be constructed by specifying appropriate index parameters, such as the algorithm type, the number of trees, etc., for quickly finding the nearest neighbors. Then, for the feature points in the three feature point sets, the constructed index can be utilized, and through the nearest neighbor search function of the fast nearest neighbor search library, the nearest neighbor points of each feature point in other feature point sets can be found. For example, a distance threshold can be set to only retain the matching points with a distance less than the threshold to improve the accuracy of the matching. Then, according to actual requirements, the obtained initial matching results can be screened and optimized. For example, the Random Sample Consensus (RANSAC) algorithm can be used to remove the mismatched points, and through random sampling and verification, the optimal set of matching points can be found to obtain the final target matching relationship.

[0063] It should be explained that the target matching relationship can include the feature point correspondence relationship and the matching distance. The feature point correspondence relationship clearly records which feature points in the three feature point sets match each other, that is, it records the index or number of the nearest neighbor feature points corresponding to each feature point in other feature point sets; the matching distance reflects the similarity degree between the matching feature points. The smaller the distance, the higher the matching degree.

[0064] In this way, by using a fast nearest neighbor search library, such as the FLANN search library, the efficiency of feature point matching can be significantly improved. The fast nearest search library adopts an optimized index structure and search algorithm, and can quickly find the nearest neighbor points in a large-scale feature point set, greatly reducing the time required for matching. For multi-dimensional image data containing three feature point sets, the fast nearest neighbor search library can efficiently complete the feature point matching task while ensuring the matching accuracy, meeting the requirements for real-time performance in practical applications.

[0065] The spatio-temporal registration unit is used to calculate the geometric transformation relationship between different images in the first multi-dimensional image data according to the target matching relationship and a preset perspective transformation method to obtain the target geometric transformation relationship; align different images in the first multi-dimensional image data according to the target geometric transformation relationship to obtain the second multi-dimensional image data; adjust the resolution of different images in the second multi-dimensional image data to a preset resolution to obtain the third multi-dimensional image data.

[0066] In specific embodiments, the coordinate points of the feature points that match each other between different images can be obtained from the target matching relationship. For example, for visible light images and thermal imaging images, the horizontal and vertical coordinates of the feature points they match in their respective image coordinate systems are extracted to form multiple sets of matching point pairs. Then, a perspective transformation method and multiple sets of matching point pairs can be preset to determine the target geometric transformation relationship. For example, assuming that the preset perspective transformation method is the direct linear transformation method, a linear equation system containing multiple equations can be constructed based on multiple sets of matching point pairs. Specifically, the perspective transformation relationship (i.e., the geometric transformation relationship) can be represented by a 3×3 homography matrix H. For each set of matching points (x1, y1) and (x2, y2), the following relationship exists:

[0067]

[0068] Regarding the 9 elements of H as unknowns, expanding the above equation can obtain 2 equations. Through multiple sets of matching point pairs, a linear equation system containing multiple equations can be constructed. Using the least squares method or other linear equation system solving methods to solve the constructed equation system, an estimated value of the homography matrix H can be obtained; the obtained homography matrix H describes the geometric transformation relationship between different images, that is, the target geometric transformation relationship, which contains various geometric operation information such as translation, rotation, scaling, and perspective transformation, and can be used to map points in one image to the corresponding positions in another image to achieve operations such as image registration.

[0069] Next, according to the target geometric transformation relationship, different images in the first multi-dimensional image data are aligned to obtain the second multi-dimensional image data. Specifically, each pixel point in each image to be aligned can be transformed and aligned according to the target geometric transformation relationship to obtain the second multi-dimensional image data. For example, for any point (x, y) in a certain image, its homogeneous coordinate is represented as (x, y, 1). By multiplying it with the homography matrix H, the transformed point (x’, y’, w’) is obtained, and then through x” = x’ / w’ and y” = y’ / w’, the finally transformed coordinates (x”, y”) are calculated; then, the resolutions of different images in the second multi-dimensional image data can be adjusted to the preset resolution to obtain the third multi-dimensional image data.

[0070] The three-dimensional modeling and stereoscopic unit is used to construct a three-dimensional model of the target main power transformation equipment according to the preset multi-view stereoscopic vision algorithm and the third multi-dimensional image data to obtain the target three-dimensional model.

[0071] In a specific embodiment, the spatial depth information of each image feature point in the third multi-dimensional image data can be calculated by a preset multi-view stereo vision algorithm to obtain a plurality of spatial depth information. The plurality of spatial depth information is matched according to the target matching relationship, and the matched spatial depth information is converted into discrete point clouds in a three-dimensional space to form a dense spatial coordinate set on the surface of the device. Then, noise points can be filtered out and missing areas can be filled, and the discrete point clouds are converted into a continuous three-dimensional model surface through a surface reconstruction algorithm (such as triangular meshing) to obtain the target three-dimensional model.

[0072] The multi-source data fusion unit is configured to determine the visible light texture data corresponding to the visible light image; determine the infrared temperature distribution data corresponding to the thermal imaging image; and map the visible light texture data and the infrared temperature distribution data to the surface of the target three-dimensional model to obtain the target fusion image.

[0073] In a specific embodiment, to determine the visible light texture data corresponding to the visible light image, specifically, the visible light image can be first denoised. Methods such as Gaussian filtering and median filtering can be used to remove the noise points in the image to improve the accuracy of subsequent texture extraction. Then, a texture feature extraction algorithm can be used to extract the visible light texture data, such as the gray-level co-occurrence matrix (GLCM), local binary pattern (LBP), etc. Taking GLCM as an example, by calculating the occurrence frequency of pixel pairs with different gray levels in the image, texture feature parameters such as contrast, entropy, energy, and correlation are obtained, and these parameters constitute the texture data of the visible light image.

[0074] Next, to determine the infrared temperature distribution data corresponding to the thermal imaging image, specifically, the thermal imaging image can be first radiometrically calibrated, and the pixel values are converted into actual temperature values through radiometric calibration. Then, for each pixel point in the thermal imaging image, its corresponding temperature value can be calculated according to a preset calibration model, thereby obtaining the temperature infrared distribution data of the entire thermal imaging image.

[0075] Finally, the visible light texture data and the infrared temperature distribution data can be mapped to the surface of the target three-dimensional model, and the image of the target three-dimensional model at this time is obtained to obtain the target fusion image.

[0076] In this way, by calculating the geometric transformation relationship through the target matching relationship and the perspective transformation method, different images can be accurately aligned. In the monitoring of main substation equipment, different types of images may have deviations due to factors such as the position and angle of the shooting equipment. After accurate alignment, it can ensure that the positions of the same equipment area in different images are consistent, facilitating subsequent comprehensive analysis. In addition, by using the preset multi-view stereo vision algorithm and the third multi-dimensional image data to construct the target three-dimensional model, the structure of the main substation equipment can be comprehensively displayed from multiple angles and dimensions. This helps the staff to more intuitively understand the appearance, shape and spatial layout of the equipment, and timely discover potential physical defects, such as equipment deformation and component loss.

[0077] Optionally, in terms of adjusting the resolution of different images in the second multi-dimensional image data to the preset resolution to obtain the third multi-dimensional image data, the spatio-temporal registration unit is specifically used for:

[0078] A1. Determine the resolution of each image in the second multi-dimensional image data to obtain three resolutions;

[0079] A2. Determine the difference between each resolution in the three resolutions and the preset resolution to obtain three differences;

[0080] A3. Determine the resampling direction corresponding to each image in the second multi-dimensional image data according to the three differences to obtain three resampling directions; each resampling direction includes upsampling or downsampling;

[0081] A4. Determine the image feature information of each image in the second multi-dimensional image data to obtain three image feature information;

[0082] A5. Determine three resampling algorithms according to the three resampling directions and the three image feature information;

[0083] A6. Based on the three resampling algorithms, adjust the image resolution in the second multi-dimensional image data to the preset resolution to obtain the third multi-dimensional image data.

[0084] In the embodiments of the present application, the resolution of each image in the second multi-dimensional image data can be determined first to obtain three resolutions. For example, image software can be used to read the resolution of each image in the second multi-dimensional image data to obtain three resolutions. Then, each resolution among the three resolutions can be subtracted by a preset resolution to obtain three differences. Next, the resampling direction corresponding to each image in the second multi-dimensional image data can be determined based on the three differences to obtain three resampling directions. For example, assuming that the first difference (any one of the three differences) is greater than 0, this indicates that the resolution of the corresponding image is greater than the preset resolution and the resolution of this image needs to be reduced. At this time, the resampling direction corresponding to the first difference is downsampling. On the contrary, if the first difference is less than 0, the resampling direction corresponding to the first difference is upsampling.

[0085] Next, the image feature information of each image in the second multi-dimensional image data can be determined to obtain three pieces of image feature information. Specifically, for each image, its color histogram can be calculated. The color space of the image is divided into several intervals, and the number of pixels in each interval is counted to obtain the color histogram feature (i.e., the image feature information). The color moment can also be calculated, and statistical quantities such as the mean, variance, and third-order moment of the image color are used to describe the color characteristics of the image, that is, the image feature information. Thus, three pieces of image feature information are obtained.

[0086] Next, three resampling algorithms can be determined based on the three resampling directions and the three pieces of image feature information. Specifically, three resampling algorithm sets can be determined first according to the three resampling directions. Then, appropriate resampling algorithms are selected from the three resampling algorithm sets based on the three pieces of image feature information to obtain three resampling algorithms. For example, if the resampling direction is upsampling, that is, the number of pixels in the image needs to be increased, the corresponding resampling algorithm set can include the bilinear interpolation algorithm or the bicubic interpolation algorithm. The bilinear interpolation algorithm is simple to calculate and can provide a good visual effect; the bicubic interpolation algorithm takes into account more neighborhood pixel information and can better retain image details, which is suitable for scenarios with high requirements for image quality. If the image feature information shows that the image has rich texture, in order to better retain texture details, the bicubic interpolation algorithm is preferably selected during upsampling, and the Gaussian sampling algorithm that can better retain edges and texture can be considered during downsampling. Finally, the corresponding images in the second multi-dimensional image data can be processed based on the three resampling algorithms to adjust their resolutions to the preset resolution. Thus, the third multi-dimensional image data is obtained.

[0087] Thus, by determining the resampling direction (upsampling or downsampling) according to the difference between the current resolution and the preset resolution of the image, unnecessary information loss or redundancy can be avoided. For images with a resolution lower than the preset value, upsampling can increase the detailed information of the image and make it consistent with other images in terms of information richness; for images with a resolution higher than the preset value, downsampling can reduce the data volume, lower the computational complexity, and at the same time remove some unnecessary noise and details.

[0088] Optionally, the system is further specifically configured to:

[0089] B1. Obtain the fault detection accuracy requirement corresponding to the target main substation equipment;

[0090] B2. Determine the first resolution corresponding to the fault detection accuracy requirement;

[0091] B3. Obtain the maximum resolution and equipment performance parameters of the equipment corresponding to the system;

[0092] B4. Determine the second resolution corresponding to the equipment performance parameters;

[0093] B5. Determine the first weight corresponding to the first resolution and the second weight corresponding to the second resolution; the sum of the first weight and the second weight is 1;

[0094] B6. Determine the target resolution according to the first weight, the second weight, the first resolution, and the second resolution;

[0095] B7. When the target resolution is less than the maximum resolution of the equipment, determine the preset resolution according to the target resolution;

[0096] B8. When the target resolution is not less than the maximum resolution of the equipment, determine the preset resolution according to the maximum resolution of the equipment.

[0097] In the embodiments of the present application, the equipment performance parameters may include at least one of the following: the main frequency of the central processing unit, the core frequency of the graphics processing unit, the capacity, etc., which are not limited herein.

[0098] In a specific embodiment, the fault detection accuracy requirement corresponding to the target substation main equipment can be obtained. Specifically, the equipment manual of the target substation main equipment can be obtained, and the equipment manual has clear provisions on the fault detection indicators and accuracy of the target substation main equipment. By consulting the equipment manual, the basic accuracy requirements applicable to the target substation main equipment can be obtained, that is, the fault detection accuracy requirement. Alternatively, the fault detection accuracy requirement can be manually input into the system by the staff; then, the first resolution corresponding to the fault detection accuracy requirement can be determined. Specifically, the mapping relationship between the preset detection accuracy requirement and the resolution can be pre-stored, and the first resolution corresponding to the fault detection accuracy requirement can be determined based on the mapping relationship.

[0099] Next, the maximum resolution and performance parameters of the device corresponding to the system can be obtained. Specifically, the system user manual of the system can be obtained, and the maximum resolution and performance parameters of the device can be obtained from the system user manual; further, the second resolution corresponding to the device performance parameters can be determined. Similarly, the mapping relationship between the preset performance parameters and the resolution can be pre-stored, and the second resolution corresponding to the device performance parameters can be determined based on the mapping relationship; then, the first weight corresponding to the first resolution and the second weight corresponding to the second resolution can be determined. Specifically, the performance indicators of the system when running typical tasks at the first resolution and the second resolution can be tested respectively, such as CPU occupancy, GPU power consumption, frame rate, etc. For example, when running a graphics processing software, the initial GPU usage rate is 30%, and the GPU usage rate at 2K resolution (first resolution) and 1080P resolution (second resolution) is tested. The weight is determined according to the change of the performance indicator. If the device performance at the first resolution is significantly reduced, it means that the resolution has high requirements for device performance. In order to balance performance and effect, its weight can be appropriately reduced. For example, the GPU utilization rate reaches 80% at the first resolution and 50% at the second resolution. The weight can be determined according to the magnitude of the performance change. For example, the weight can be determined by calculating the ratio of the performance index change value to the total change rate. It can be seen that the GPU utilization rate at the first resolution increases by 80%-30%=50%, and the GPU utilization rate at the second resolution increases by 50%-30%=20%. Since the performance index change value is inversely proportional to the weight, therefore:

[0100] 50% / 20%=second weight / first weight;

[0101] First weight + second weight = 1;

[0102] The above two equations can be combined to obtain the first weight and the second weight. Then, a weighted operation can be performed based on the first weight, the second weight, the first resolution and the second resolution to obtain the target resolution. When the target resolution is less than the maximum resolution of the device, the target resolution can be used as the preset resolution.

[0103] When the target resolution is not less than the maximum resolution of the device, the maximum resolution of the device can be used as the preset resolution.

[0104] In this way, by obtaining the fault detection accuracy requirements corresponding to the target main power transformation equipment to determine the first resolution, it can ensure that the resolution can meet the basic requirements for detecting fault characteristics, avoid missing key fault information due to too low resolution, and improve the accuracy and reliability of fault detection; in addition, determining the weights of the first resolution and the second resolution and determining the target resolution accordingly can find the best balance point between fault detection accuracy and device performance. It not only ensures the accuracy of fault detection but also can give full play to the device performance and improve the overall operation efficiency of the system.

[0105] Among them, the intelligent analysis module is used to perform fault identification on the target main power transformation equipment according to the target fusion image and the target device data to obtain the target fault identification result;

[0106] Optionally, for performing fault identification on the target main power transformation equipment according to the target fusion image and the target device data to obtain the target fault identification result, please refer to Figure 4 , Figure 4 is a schematic structural diagram of an intelligent analysis module provided by an embodiment of the present application. As Figure 4 shown, the intelligent analysis module includes: a fault detection unit, a status prediction unit, and a knowledge graph unit; where:

[0107] The fault detection unit is used to extract the feature data of the target fusion image to obtain the first feature data; extract the feature data of the target device data to obtain the second feature data; and perform reasoning according to the first feature data and the second feature data through a preset fault identification model to obtain the first identification result;

[0108] In the embodiment of the present application, the preset fault identification model can be preset or default in advance.

[0109] In a specific embodiment, an edge feature extraction algorithm can be used to perform feature processing on the target fusion image to obtain the first feature data. Then, since most of the data in the target device data are numerical, such as parameters such as the voltage, current, and temperature of the device, the statistical feature extraction can be directly performed on these original numerical values to obtain the second feature data, such as calculating the mean, variance, maximum value, minimum value, median, etc. For example, for the temperature data of the device, calculating its mean can reflect the average temperature state of the device within a period of time, and the variance can reflect the temperature fluctuation situation; finally, the first feature data and the second feature data can be input into the preset fault identification model to obtain the first identification result.

[0110] A state prediction unit is used to preprocess the target fusion image and target device data to obtain preprocessed data. The preprocessing includes at least one of the following: normalization, feature extraction, and time window division. The preprocessed data is input into a preset long short-term memory network model, and the future operating conditions of the target main substation equipment are predicted through the preset long short-term memory network model, resulting in a first prediction result. The first prediction result may include: predicted values of future operating parameters, future failure probabilities, remaining life assessment values, etc., which are not limited here.

[0111] In the embodiments of the present application, the preset long short-term memory network model can be preset in advance or default.

[0112] In a specific embodiment, the target fusion image and target device data can be preprocessed first to obtain preprocessed data. Then, the preprocessed data can be input into a preset long short-term memory network model, and the future operating conditions of the target main substation equipment are predicted through the preset long short-term memory network model, resulting in a first prediction result.

[0113] A knowledge graph unit is used to obtain the historical device data of the target main substation equipment; according to the historical device data and a preset expert rule base, a knowledge graph model including the association relationships between equipment, faults, and maintenance strategies is constructed; when the fault detection unit performs reasoning, the knowledge graph model is used to search and match according to the target device data to obtain a fault diagnosis logical reasoning chain;

[0114] In the embodiments of the present application, the preset expert rule base can be preset in advance or default. The preset expert rule base may include equipment operation parameter rules, fault feature rules, protection action rules, maintenance rules, operating environment rules, operation sequence rules, etc., which are not limited here.

[0115] In a specific embodiment, the historical device data of the target main substation equipment can be obtained. Specifically, authorization from the target power system can be obtained to access the database of the target power system, and relevant data of the target main substation equipment can be queried from the database to obtain the historical device data.

[0116] Next, a knowledge graph model including the association relationships among devices, faults, and maintenance strategies can be constructed based on historical device data and a preset expert rule library. Specifically, entities are extracted from the historical device data and the preset expert rule library, including device entities (such as transformers, circuit breakers, etc.), fault entities (such as winding short circuits, over-high oil temperature), and maintenance strategy entities (such as oil sample analysis, component replacement), etc. There is no limitation here, and named entity recognition technology or rule-based methods can be used to complete entity recognition; next, the relationships among devices, faults, and maintenance strategies can be determined. For example, according to historical data, it is found that a certain device fault is associated with a specific device, and at the same time, according to the expert rule library, the maintenance strategy corresponding to this fault is clarified. Methods such as rule matching and machine learning classification can be used to extract these relationships. For example, the relationship between "transformer" and "winding short circuit" is "fault occurs", and the relationship between "winding short circuit" and "replace winding" is "corresponding maintenance strategy"; then, the identified entities and the extracted relationships can be represented in the form of a graph, and a graph database can be used to store and manage the knowledge graph. Each entity serves as a node, and the relationship between entities serves as an edge, thereby constructing a knowledge graph model including the association relationships among devices, faults, and maintenance strategies; when the fault detection unit performs reasoning, it can search and match according to the target device data through the knowledge graph model to obtain a fault diagnosis logical reasoning chain.

[0117] Finally, the intelligent analysis module can fuse the first recognition result, the first prediction result, and the fault diagnosis logical reasoning chain to obtain the target fault recognition result.

[0118] In this way, the fault detection unit comprehensively utilizes the features of the target fusion image and the target device data and performs reasoning through a preset fault recognition model. The fusion of multi-source data can reflect the device state from different perspectives, avoid the limitations of a single data source, and thus improve the accuracy and reliability of fault detection. In addition, the state prediction unit preprocesses the data and uses a preset long short-term memory network model to predict the future operation conditions of the target main substation equipment, and can obtain information such as predicted values of future operation parameters of the equipment, future fault probabilities, and remaining life evaluation values in advance. This helps the staff to make preparations in advance, take preventive maintenance measures, and reduce the equipment fault downtime.

[0119] Among them, the decision-making and alarm module is used to determine the target maintenance plan and the target alarm operation corresponding to the target fault recognition result.

[0120] Optionally, for determining the target maintenance plan and the target alarm operation corresponding to the target fault recognition result, please refer to Figure 5 , Figure 5 which is a structural schematic diagram of a decision-making and alarm module provided by an embodiment of the present application. As Figure 5As shown in the figure, the decision-making alarm module includes: a real-time monitoring unit, a maintenance strategy formulation unit, and an alarm prompt unit; among which:

[0121] The real-time monitoring unit is used to monitor the operating status of the target main substation equipment and the output result of the intelligent analysis module in real time, so as to update the equipment operating status of the target main substation equipment in the system, and regularly store the collected equipment operating data, the output result of the intelligent analysis module, and the update record of the equipment operating status, etc. into the database of the system.

[0122] The maintenance strategy formulation unit is used to determine the target fault type and fault severity according to the target fault identification result; evaluate the priority of the target fault type through the analytic hierarchy process to obtain the target priority; and determine the target maintenance plan according to the target priority, the fault diagnosis logic reasoning chain, and the preset expert rule base.

[0123] In a specific embodiment, the fault-related information can be sorted out from the target fault identification result. For example, the fault occurrence location, fault abnormal value, etc. are determined. The target fault type is determined according to the fault occurrence location, and the fault severity is determined according to the abnormal situation of relevant parameters. For example, for high-voltage switchgear, "contact part" corresponds to "contact overheating fault" and "contact wear fault". In addition, the mapping relationship between the preset abnormal value and the severity can be stored in advance, and the fault severity corresponding to the fault abnormal value is determined based on this mapping relationship. For example, for the transformer oil temperature parameter, when the oil temperature abnormal value exceeds the normal upper limit by less than 5 degrees Celsius, it is a "minor" fault; when it exceeds 5-10 degrees Celsius, it is a "moderate" fault; when it exceeds 10-20 degrees Celsius, it is a "severe" fault; when it exceeds 20 degrees Celsius or more, it is an "extremely severe" fault.

[0124] Next, the priority of the target fault type can be evaluated through the Analytic Hierarchy Process (AHP) to obtain the target priority. Since the AHP is a conventional technology, it will not be elaborated here. Further, based on the target priority, the fault diagnosis logic reasoning chain, and the preset expert rule base, the target maintenance plan can be determined. Specifically, the fault types can be sorted according to the target priority, and faults with high priorities can be given priority attention. For example, if the priority of the transformer winding short-circuit fault is higher than that of the iron core grounding fault, then the equipment corresponding to the winding short-circuit fault will be taken as the primary maintenance object, and the list of equipment that needs to be repaired immediately will be determined to ensure that the maintenance resources are preferentially invested in the most critical fault handling. Then, along the fault diagnosis logic reasoning chain, starting from the identified fault phenomena, the root causes leading to the faults can be gradually analyzed in depth. For example, for the overheating fault of the switchgear contact, it may be found through the reasoning chain that it is due to poor contact of the contact, and the poor contact is caused by the loosening of the bolt due to long-term vibration. Based on the reasoning chain, the impacts that the faults may have on other components or systems of the equipment can be clarified. According to the fault type, cause, and influence range, appropriate maintenance methods can be selected from the preset expert rule base to obtain at least one maintenance method. Referring to the preset expert rule base, the time requirements and required resources for the maintenance work can be determined. According to the preset maintenance strategy (for example, performing maintenance from easy to difficult), the maintenance tasks and at least one maintenance method can be arranged in sequence to form a detailed maintenance process. Thus, the target maintenance plan can be obtained.

[0125] An alarm prompt unit is used to determine the target alarm operation according to the fault severity. Specifically, the mapping relationship between the preset severity and the alarm operation can be pre-stored, and the target alarm operation corresponding to the fault severity can be determined based on this mapping relationship.

[0126] In this way, through: the real-time monitoring unit can monitor the operation status of the target main substation equipment and the output results of the intelligent analysis module in real time, and can capture the subtle changes and abnormal signs in the equipment operation in a timely manner, enabling the staff to understand the true situation of the equipment at any time, providing an accurate basis for subsequent decision-making, and avoiding the expansion of faults caused by information lag. In addition, by determining the target maintenance plan based on the target priority, the fault diagnosis logic reasoning chain, and the preset expert rule base, a comprehensive, scientific, and feasible maintenance plan can be formulated in combination with the actual fault situation and expert experience. This plan covers aspects such as maintenance methods, processes, personnel arrangements, and time planning, ensuring the orderly progress of the maintenance work, improving the maintenance quality, and effectively restoring the equipment performance.

[0127] Optionally, the system further includes: a visualization display module, please refer to Figure 6 , Figure 6 is a schematic structural diagram of a visualization display module provided by an embodiment of the present application, as Figure 6As shown in the figure, the visualization display module includes: a dynamic display unit, an interactive operation unit, and a multi-terminal adaptation unit; where:

[0128] The dynamic display unit is used to render the target 3D model in real time through a preset 3D engine, and, according to the real-time operating status of the target main substation equipment, control the dynamic demonstration of the operating status and fault evolution process of the target 3D model; among them, the preset 3D engine can be preset in advance or by default.

[0129] The interactive operation unit is used to control the target 3D model based on a 3D interaction framework through a preset interaction method; the preset interaction method includes at least one of the following: mouse, touch screen, gesture;

[0130] The multi-terminal adaptation unit is used to configure the display requirements of at least one terminal device of the user through cross-platform rendering technology. For example, it can adapt the screen size for terminal device screens of different sizes, such as mobile phones (small screens), tablets (medium screens), and computer monitors (large screens). Taking a substation equipment operation and maintenance management system as an example, on a mobile phone, the cross-platform rendering technology can adjust the system interface layout, and centrally display the main equipment status information, key operation buttons, etc. at appropriate positions on the screen for convenient single-handed operation by the user; on a tablet, the interface layout can be more abundant, showing more device data charts and detailed information; on a computer monitor, all function modules of the system and a large amount of device operation data can be presented in a more spacious layout, and multi-window display is supported, facilitating the user to view information of different devices or perform different operations simultaneously.

[0131] It can be seen that the intelligent maintenance system based on image analysis described in this application includes: a data acquisition module, a data fusion module, an intelligent analysis module, and a decision-making and alarm module, where: The data acquisition module is used to collect multi-dimensional image data and device data of the target main substation equipment in the target power system to obtain the first multi-dimensional image data and target device data; the data fusion module is used to fuse the images in the first multi-dimensional image data to obtain a target fusion image; the target fusion image is a three-dimensional stereoscopic image; the intelligent analysis module is used to perform fault identification on the target main substation equipment according to the target fusion image and the target device data to obtain a target fault identification result; the decision-making and alarm module is used to determine the target maintenance plan and target alarm operation corresponding to the target fault identification result; thus, fault identification is performed based on the target fusion image and the target device data, and by combining these two pieces of information, it is possible to more accurately judge whether it is an external fault or an internal fault of the device. Moreover, the two aspects of information can also mutually verify and confirm the fault, thereby more precisely identifying the fault type, location, and severity, and further improving the accuracy of fault diagnosis of the main substation equipment.

[0132] Please refer to Figure 7 ,Figure 7 It is a flowchart of an intelligent maintenance method based on image analysis provided by an embodiment of the present application; the method may include the following steps:

[0133] S701. Collect multi-dimensional image data and device data of a target main substation equipment in the power system to obtain first multi-dimensional image data and target device data;

[0134] S702. Fuse the images in the first multi-dimensional image data to obtain a target fused image; the target fused image is a three-dimensional stereoscopic image;

[0135] S703. Perform fault identification on the target main substation equipment according to the target fused image and the target device data to obtain a target fault identification result;

[0136] S704. Determine a target maintenance plan and a target alarm operation corresponding to the target fault identification result.

[0137] In specific implementation, the intelligent maintenance method based on image analysis described in the embodiment of the present invention may also include other implementation manners described in the intelligent maintenance system based on image analysis provided by the embodiment of the present invention, which will not be elaborated herein.

[0138] Please refer to Figure 8 , Figure 8 It is a schematic structural diagram of an electronic device provided by an embodiment of the present application. The electronic device may include a processor, a memory, a communication interface, and one or more programs. The processor, the memory, and the communication interface may be connected to each other through a bus; the above one or more programs are stored in the above memory and are configured to be executed by the above processor; in the embodiment of the present application, the above program includes parts or all of the steps for executing the above intelligent maintenance method based on image analysis.

[0139] The embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program for electronic data exchange, and the computer program enables a computer to execute parts or all of the steps of the above intelligent maintenance method based on image analysis. The above computer includes an electronic device.

[0140] The embodiment of the present application also provides a computer program product. The above computer program product includes a non-transitory computer-readable storage medium storing a computer program. The above computer program is operable to enable a computer to execute parts or all of the steps of the above intelligent maintenance method based on image analysis. The computer program product may be a software installation package, and the above computer includes an electronic device.

[0141] It should be noted that, for the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that this application is not limited by the described action sequence, because according to this application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0142] In the above embodiments, the descriptions of the respective embodiments have their own focuses. For the parts not detailed in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0143] In several embodiments provided by this application, it should be understood that the disclosed device can be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the above division of units is only a logical function division. In actual implementation, there can be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection to each other can be through some interfaces. The indirect coupling or communication connection of the device or unit can be in electrical or other forms.

[0144] Those of ordinary skill in the art can understand all or part of the processes in the above method embodiments. These processes can be completed by relevant hardware instructed by a computer program. The program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the above method embodiments. The foregoing storage media include: ROM or random access memory RAM, magnetic disk, or optical disc and other media that can store program codes.

[0145] The steps of the methods or algorithms described in the embodiments of this application can be implemented in a hardware manner or by a processor executing software instructions. The software instructions can be composed of corresponding software modules. The software modules can be stored in RAM, flash memory, ROM, EPROM, electrically erasable programmable read-only memory (EEPROM), registers, hard disk, removable hard disk, compact disc read-only memory (CD-ROM), or any other form of storage medium well-known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC. In addition, the ASIC can be located in a terminal device or a management device. Of course, the processor and the storage medium can also exist as discrete components in the terminal device or the management device.

[0146] Those skilled in the art should be able to realize that in one or more of the above examples, the functions described in the embodiments of the present application can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part.

[0147] The above computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from a website, computer, server, or data center to another website, computer, server, or data center by wire (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wirelessly (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more integrated available media.

[0148] Among them, the available medium can be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium (such as a digital video disc (DVD)), or a semiconductor medium (such as a solid state disk (SSD)), etc.

[0149] Each device and product described in the above embodiments, and each module / unit included therein, may be a software module / unit, a hardware module / unit, or may be partly a software module / unit and partly a hardware module / unit. For example, for each device and product applied to or integrated with a chip, each module / unit included therein may be implemented in the form of hardware such as circuits, or at least some of the modules / units may be implemented in the form of software programs that run on a processor integrated within the chip, and the remaining (if any) part of the modules / units may be implemented in the form of hardware such as circuits; for each device and product applied to or integrated with a chip module, each module / unit included therein may be implemented in the form of hardware such as circuits, and different modules / units may be located in the same component (such as a chip, a circuit module, etc.) or different components of the chip module, or at least some of the modules / units may be implemented in the form of software programs that run on a processor integrated within the chip module, and the remaining (if any) part of the modules / units may be implemented in the form of hardware such as circuits; for each device and product applied to or integrated with a terminal device, each module / unit included therein may be implemented in the form of hardware such as circuits, and different modules / units may be located in the same component (such as a chip, a circuit module, etc.) or different components within the terminal device, or at least some of the modules / units may be implemented in the form of software programs that run on a processor integrated within the terminal device, and the remaining (if any) part of the modules / units may be implemented in the form of hardware such as circuits.

[0150] The specific embodiments described above further elaborate on the purpose, technical solutions, and beneficial effects of the embodiments of the present application. It should be understood that the above description is only the specific embodiments of the embodiments of the present application and is not used to limit the protection scope of the embodiments of the present application. Any modifications, equivalent replacements, improvements, etc. made on the basis of the technical solutions of the embodiments of the present application shall be included within the protection scope of the embodiments of the present application.

Claims

1. An intelligent maintenance system based on image analysis, characterized in that, The system includes: a data acquisition module, a data fusion module, an intelligent analysis module, and a decision-making and alarm module, where: The data acquisition module is configured to acquire multi-dimensional image data and equipment data of target main substation equipment in a target power system, and obtain first multi-dimensional image data and target equipment data; The data fusion module is configured to fuse the images in the first multi-dimensional image data to obtain a target fused image; the target fused image is a three-dimensional stereoscopic image; The intelligent analysis module is configured to perform fault identification on the target main substation equipment according to the target fused image and the target equipment data, and obtain a target fault identification result; The decision-making and alarm module is configured to determine a target maintenance plan and a target alarm operation corresponding to the target fault identification result.

2. The system according to claim 1, characterized in that, The first multi-dimensional image data includes: visible light images, thermal imaging images, and three-dimensional space images; In terms of fusing the images in the first multi-dimensional image data to obtain a target fused image, the data fusion module includes: a feature point matching unit, a spatio-temporal registration unit, a three-dimensional modeling and stereoscopic unit, and a multi-source data fusion unit, where: The feature point matching unit is configured to process the first multi-dimensional image data according to the SIFT algorithm to obtain three feature point sets and three feature vector sets; each dimension data corresponds to a feature point set and a feature vector set; use the FLANN library to match the feature points in the three feature point sets to obtain a target matching relationship; The spatio-temporal registration unit is configured to calculate the geometric transformation relationship between different images in the first multi-dimensional image data according to the target matching relationship and a preset perspective transformation method, and obtain a target geometric transformation relationship; align different images in the first multi-dimensional image data according to the target geometric transformation relationship to obtain second multi-dimensional image data; adjust the resolution of different images in the second multi-dimensional image data to a preset resolution to obtain third multi-dimensional image data; The three-dimensional modeling and stereoscopic unit is configured to construct a three-dimensional model of the target main substation equipment according to a preset multi-view stereovision algorithm and the third multi-dimensional image data, and obtain a target three-dimensional model; The multi-source data fusion unit is configured to determine visible light texture data corresponding to the visible light image; determine infrared temperature distribution data corresponding to the thermal imaging image; map the visible light texture data and the infrared temperature distribution data to the surface of the target three-dimensional model to obtain the target fused image.

3. The system according to claim 2, characterized in that In terms of adjusting the resolution of different images in the second multi-dimensional image data to a preset resolution to obtain third multi-dimensional image data, the spatio-temporal registration unit specifically is configured to: Determine the resolution of each image in the second multi-dimensional image data to obtain three resolutions; Determine the difference between each resolution in the three resolutions and the preset resolution to obtain three differences; Determine the resampling direction corresponding to each image in the second multi-dimensional image data according to the three differences, obtaining three resampling directions; each resampling direction includes upsampling or downsampling; Determine the image feature information corresponding to each image in the second multi-dimensional image data, obtaining three pieces of image feature information; Determine three resampling algorithms according to the three resampling directions and the three pieces of image feature information; Adjust the image resolution in the second multi-dimensional image data to the preset resolution based on the three resampling algorithms, obtaining the third multi-dimensional image data.

4. The system according to claim 3, wherein, The system is further specifically configured to: Obtain the fault detection accuracy requirement corresponding to the target main substation equipment; Determine the first resolution corresponding to the fault detection accuracy requirement; Obtain the maximum resolution and equipment performance parameters of the equipment corresponding to the system; Determine the second resolution corresponding to the equipment performance parameters; Determine the first weight corresponding to the first resolution and the second weight corresponding to the second resolution; The sum of the first weight and the second weight is 1; Determine the target resolution according to the first weight, the second weight, the first resolution and the second resolution; When the target resolution is less than the maximum resolution of the equipment, determine the preset resolution according to the target resolution; When the target resolution is not less than the maximum resolution of the equipment, determine the preset resolution according to the maximum resolution of the equipment.

5. The system according to any one of claims 1-4, characterized in that, In terms of performing fault identification on the target main substation equipment according to the target fusion image and the target equipment data to obtain a target fault identification result, the intelligent analysis module includes: a fault detection unit, a state prediction unit, and a knowledge graph unit; wherein: The fault detection unit is configured to extract the feature data of the target fusion image to obtain first feature data; extract the feature data of the target equipment data to obtain second feature data; perform inference according to the first feature data and the second feature data through a preset fault identification model to obtain a first identification result; The state prediction unit is configured to perform preprocessing on the target fusion image and the target equipment data to obtain preprocessed data, and the preprocessing includes at least one of the following: normalization, feature extraction, and dividing a time window; input the preprocessed data into a preset long short-term memory network model, and predict the future operation condition of the target main substation equipment through the preset long short-term memory network model, a first prediction result; the first prediction result includes: predicted future operation parameter values, future fault probabilities, remaining life evaluation values; The knowledge graph unit is configured to obtain the historical equipment data of the target main substation equipment; construct a knowledge graph model including the association relationships between equipment, faults and maintenance strategies according to the historical equipment data and a preset expert rule base; when the fault detection unit performs inference, perform search and matching according to the target equipment data through the knowledge graph model to obtain a fault diagnosis logical inference chain; The intelligent analysis module determines the target fault identification result according to the first identification result, the first prediction result, and the fault diagnosis logic inference chain.

6. The system according to claim 5, wherein In terms of determining the target maintenance plan and target alarm operation corresponding to the target fault identification result, the decision-making and alarm module includes: a real-time monitoring unit, a maintenance strategy formulation unit, and an alarm prompt unit; where: The real-time monitoring unit is configured to monitor the operating state of the target main substation equipment and the output result of the intelligent analysis module in real time, so as to update the equipment operating state of the target main substation equipment in the system. The maintenance strategy formulation unit is configured to determine the target fault type and fault severity according to the target fault identification result; evaluate the priority of the target fault type through the analytic hierarchy process to obtain the target priority; and determine the target maintenance plan according to the target priority, the fault diagnosis logic inference chain, and the preset expert rule library. The alarm prompt unit is configured to determine the target alarm operation according to the fault severity.

7. The system according to claim 2 or 3, characterized in that The system further includes: a visualization display module, and the visualization display module includes: a dynamic display unit, an interaction operation unit, and a multi-terminal adaptation unit; where: The dynamic display unit is configured to render the target three-dimensional model in real time through a preset three-dimensional engine, and control the target three-dimensional model to dynamically demonstrate the operating state and fault evolution process of the equipment according to the real-time operating state of the target main substation equipment. The interaction operation unit is configured to control the target three-dimensional model based on a three-dimensional interaction framework through a preset interaction method; the preset interaction method includes at least one of the following: mouse, touch screen, gesture. The multi-terminal adaptation unit is configured to configure the display requirements of at least one terminal device of the user through cross-platform rendering technology.

8. An intelligent maintenance method based on image analysis, characterized in that, The method includes: Collecting multi-dimensional image data and device data of the target main substation equipment in the power system to obtain first multi-dimensional image data and target device data. Fusing the images in the first multi-dimensional image data to obtain a target fused image; the target fused image is a three-dimensional stereoscopic image. Performing fault identification on the target main substation equipment according to the target fused image and the target device data to obtain a target fault identification result. Determining a target maintenance plan and a target alarm operation corresponding to the target fault identification result.

9. An electronic device, characterized in that, Including: A processor, a memory, a communication interface, and one or more programs. The one or more programs are stored in the memory and are configured to be executed by the processor, and the programs include instructions for performing the steps in the method as claimed in claim 8.

10. A computer-readable storage medium, characterized in that, Storing a computer program for electronic data exchange, wherein the computer program causes a computer to execute the method as claimed in claim 8.

Images