Power transmission and transformation equipment inspection system and method based on visible light image recognition technology

Abstract

The present application relates to the field of intelligent inspection, and specifically discloses a power transmission and transformation equipment inspection system and method based on visible light image recognition technology. The system uses real-time status images of power transmission equipment collected by inspection drones, extracts a set of state reference images labeled as normal from a background database, and uses deep learning-based image recognition and analysis algorithms to process and analyze the real-time status images and the state reference images labeled as normal. The system automatically determines whether the power transmission equipment has an abnormal state based on the comparison of the semantic similarity between the real-time status images and the state reference images labeled as normal with a predetermined threshold. In this way, real-time monitoring of the status of power transmission equipment can be achieved, and whether the power transmission equipment has an abnormal state can be accurately determined, reducing the possibility of misjudgment and missed detection, while reducing the cost of manual inspection, improving inspection efficiency, and enhancing the safety and reliability of power grid operation.

Description

Technical Field

[0001] The present application relates to the field of intelligent inspection, and more specifically, to a power transmission and transformation equipment inspection system and method based on visible light image recognition technology. Background Art

[0002] Power transmission and transformation equipment inspections refer to the regular inspection and monitoring of transmission and transformation equipment to ensure its normal operation and safety. These inspections can promptly identify potential equipment problems and hidden dangers, allowing preventive measures to be taken. This ensures stable performance of the equipment, ultimately improving the reliability and stability of the entire power supply system.

[0003] However, traditional inspections of power transmission and transformation equipment usually rely on manual inspections, which require on-site visits and visual inspections by personnel. This is time-consuming and has high labor costs. In addition, manual inspections are prone to omissions due to negligence, fatigue, and other reasons, making it difficult to achieve comprehensive real-time monitoring of equipment status. As a result, equipment failures are not discovered in a timely manner, thereby affecting the safe operation of the power grid and bringing certain safety hazards.

[0004] Therefore, a power transmission and transformation equipment inspection system based on visible light image recognition technology is desired. Summary of the Invention

[0005] In order to solve the above technical problems, the present application is proposed. The embodiment of the present application provides a power transmission and transformation equipment inspection system and method based on visible light image recognition technology, which uses real-time status images of power transmission equipment collected by inspection drones, and extracts a set of status reference images marked as normal from the background database, and uses image recognition and analysis algorithms based on deep learning to perform image processing and analysis on the real-time status images and the status reference images marked as normal, so as to automatically determine whether the power transmission equipment has an abnormal state based on the comparison between the semantic similarity between the real-time status image and the status reference image marked as normal and a predetermined threshold. In this way, real-time monitoring of the status of the power transmission equipment can be achieved, and whether the power transmission equipment has an abnormal state can be accurately determined, reducing the possibility of misjudgment and missed detection, while reducing the cost of manual inspection, improving inspection efficiency, and enhancing the safety and reliability of power grid operation.

[0006] According to one aspect of the present application, a power transmission and transformation equipment inspection system based on visible light image recognition technology is provided, which includes:

[0007] The real-time status acquisition module of the power transmission equipment is used to obtain the real-time status images of the power transmission equipment collected by the inspection drone;

[0008] A state reference image acquisition module is used to extract a set of state reference images marked as normal from a background database;

[0009] a state reference feature extraction module, configured to extract image features from each state reference image in the set of state reference images marked as being in a normal state to obtain a set of state reference feature maps;

[0010] a feature screening module, configured to pass each state reference feature graph in the set of state reference feature graphs through a feature distiller based on a feature screening network to obtain a set of distilled state reference feature vectors;

[0011] A feature joint clustering analysis module, configured to pass the set of distillation state reference feature vectors through a joint clustering analysis network to obtain a cluster characterization feature matrix of a normal state of the power transmission equipment;

[0012] A power transmission equipment real-time state feature extraction module, configured to extract image features from the real-time state image to obtain a power transmission equipment real-time state feature matrix;

[0013] A similarity calculation module, configured to calculate a hash similarity between the normal state cluster representation feature matrix of the power transmission device and the real-time state feature matrix of the power transmission device;

[0014] The power transmission device abnormality judgment module is used to determine whether the power transmission device has an abnormal state based on the comparison between the hash similarity and a predetermined threshold.

[0015] According to another aspect of the present application, a method for inspecting power transmission and transformation equipment based on visible light image recognition technology is provided, which includes:

[0016] Obtain real-time status images of power transmission equipment collected by inspection drones;

[0017] Extracting a set of reference images marked as being in a normal state from a background database;

[0018] Performing image feature extraction on each state reference image in the set of state reference images marked as being in a normal state to obtain a set of state reference feature maps;

[0019] Passing each state reference feature graph in the set of state reference feature graphs through a feature distiller based on a feature screening network to obtain a set of distilled state reference feature vectors;

[0020] Passing the set of distillation state reference feature vectors through a joint cluster analysis network to obtain a normal state cluster characterization feature matrix of the power transmission equipment;

[0021] Performing image feature extraction on the real-time state image to obtain a real-time state feature matrix of the power transmission equipment;

[0022] Calculating the hash similarity between the normal state cluster representation feature matrix of the power transmission device and the real-time state feature matrix of the power transmission device;

[0023] Based on the comparison between the hash similarity and a predetermined threshold, it is determined whether the power transmission device has an abnormal state.

[0024] Compared to the prior art, the present application provides a power transmission and transformation equipment inspection system and method based on visible light image recognition technology. This system uses real-time status images of power transmission equipment collected by inspection drones, extracts a set of state reference images labeled as normal from a backend database, and utilizes a deep learning-based image recognition and analysis algorithm to process and analyze the real-time status images and the state reference images labeled as normal. This system automatically determines whether the power transmission equipment is in an abnormal state based on a comparison of the semantic similarity between the real-time status images and the state reference images labeled as normal and a predetermined threshold. In this way, real-time monitoring of the power transmission equipment status can be achieved, and accurate determination of whether the power transmission equipment is in an abnormal state can be made, reducing the possibility of misjudgment and missed detection, while reducing the cost of manual inspections, improving inspection efficiency, and enhancing the safety and reliability of power grid operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The above and other purposes, features, and advantages of the present application will become more apparent through a more detailed description of the embodiments of the present application in conjunction with the accompanying drawings. The accompanying drawings are intended to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the drawings, the same reference numerals generally represent the same components or steps.

[0026] Figure 1 2 is a block diagram of a power transmission and transformation equipment inspection system based on visible light image recognition technology according to an embodiment of the present application.

[0027] Figure 2 Schematic diagram of the architecture of a power transmission and transformation equipment inspection system based on visible light image recognition technology according to an embodiment of the present application.

[0028] Figure 3 This is a block diagram of a feature joint clustering analysis module in a power transmission and transformation equipment inspection system based on visible light image recognition technology according to an embodiment of the present application.

[0029] Figure 4 This is a block diagram of a training module in a power transmission and transformation equipment inspection system based on visible light image recognition technology according to an embodiment of the present application.

[0030] Figure 5Flowchart of a method for inspecting power transmission and transformation equipment based on visible light image recognition technology according to an embodiment of the present application. DETAILED DESCRIPTION

[0031] The following describes embodiments of the present disclosure in more detail with reference to the accompanying drawings. While the drawings illustrate certain embodiments of the present disclosure, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of protection of the present disclosure.

[0032] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in a different order and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0033] In the description of the embodiments of the present disclosure, the term "including" and similar terms should be understood as open inclusion, i.e., "including but not limited to." The term "based on" should be understood as "based at least in part on." The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment." The terms "first," "second," etc. may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0034] It should be noted that the modifications of "one" and "multiple" mentioned in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, they should be understood as "one or more".

[0035] Transmission equipment, including transmission lines, towers, insulator strings, and air lines, is the equipment used to transmit electrical energy within a power system. These devices boost the voltage of electricity generated by generators and connect it to transmission lines through control devices such as circuit breakers, enabling long-distance transmission of electrical energy. Together with substation equipment, transmission equipment forms the transmission network, a crucial energy artery in modern society.

[0036] Power transmission and transformation equipment inspections refer to the regular inspection and monitoring of transmission and transformation equipment to ensure its normal operation and safety. These inspections can promptly identify potential equipment problems and hidden dangers, allowing preventive measures to be taken. This ensures stable performance of the equipment, ultimately improving the reliability and stability of the entire power supply system.

[0037] However, traditional inspections of power transmission and transformation equipment usually rely on manual inspections, which require on-site visits and visual inspections by personnel. This is time-consuming and has high labor costs. In addition, manual inspections are prone to omissions due to negligence, fatigue, and other reasons, making it difficult to achieve comprehensive real-time monitoring of equipment status. As a result, equipment failures are not discovered in a timely manner, thereby affecting the safe operation of the power grid and bringing certain safety hazards.

[0038] It should be understood that visible light image recognition technology uses image data within the visible spectrum to identify and analyze objects. By processing and analyzing visible light images, the system can identify, classify, and locate different target objects or scenes, thereby achieving automated monitoring and diagnosis. During power transmission and transformation equipment inspections, this technology can be used to automatically detect and identify equipment and defects in images, thereby improving inspection efficiency and accuracy. For example, drone-mounted cameras can automatically capture images of transmission lines, then use image recognition technology to identify and annotate damage or anomalies on the lines, such as damaged insulators or foreign objects hanging on conductors. It can also monitor equipment operating status in real time, identify anomalies, and take timely action. Therefore, visible light image recognition can reduce the workload of manual inspections while improving inspection quality and safety.

[0039] Based on this, the technical concept of this application is to use real-time status images of power transmission equipment collected by inspection drones, extract a set of status reference images labeled as normal from a background database, and use deep learning-based image recognition and analysis algorithms to perform image processing and analysis on the real-time status images and the status reference images labeled as normal. The system automatically determines whether the power transmission equipment has an abnormal status based on the comparison between the semantic similarity between the real-time status images and the status reference images labeled as normal and a predetermined threshold. In this way, the status of the power transmission equipment can be monitored in real time and accurately determined whether the power transmission equipment has an abnormal status, reducing the possibility of misjudgment and missed detection, while reducing the cost of manual inspections, improving inspection efficiency, and enhancing the safety and reliability of power grid operations.

[0040] Figure 1 2 is a block diagram of a power transmission and transformation equipment inspection system based on visible light image recognition technology according to an embodiment of the present application. Figure 2 FIG. 1 is a schematic diagram of the architecture of a power transmission and transformation equipment inspection system based on visible light image recognition technology according to an embodiment of the present application. Figure 1 and Figure 2As shown, according to an embodiment of the present application, a power transmission and transformation equipment inspection system 100 based on visible light image recognition technology includes: a power transmission equipment real-time state acquisition module 110, which is used to acquire a real-time state image of the power transmission equipment acquired by an inspection drone; a state reference image acquisition module 120, which is used to extract a set of state reference images marked as normal from a background database; a state reference feature extraction module 130, which is used to extract image features from each state reference image in the set of state reference images marked as normal to obtain a set of state reference feature graphs; a feature screening module 140, which is used to extract each state reference feature graph in the set of state reference feature graphs through a feature distiller based on a feature screening network to obtain a state reference feature graph. A set of distilled state reference feature vectors is obtained; a feature joint cluster analysis module 150 is used to apply the set of distilled state reference feature vectors to a joint cluster analysis network to obtain a normal state cluster representation feature matrix of the power transmission equipment; a real-time state feature extraction module 160 is used to extract image features from the real-time state image to obtain a real-time state feature matrix of the power transmission equipment; a similarity calculation module 170 is used to calculate the hash similarity between the normal state cluster representation feature matrix of the power transmission equipment and the real-time state feature matrix of the power transmission equipment; and a power transmission equipment abnormality judgment module 180 is used to determine whether the power transmission equipment has a state abnormality based on a comparison between the hash similarity and a predetermined threshold.

[0041] In an embodiment of the present application, the real-time status acquisition module 110 of the power transmission equipment is used to obtain the real-time status image of the power transmission equipment collected by the inspection drone. The status reference image acquisition module 120 is used to extract a set of status reference images marked as normal from the background database. It should be understood that, considering that the real-time status image reflects various status information of the equipment, for example, the physical condition of the equipment (such as damage, corrosion, deformation), the operating condition of the equipment (such as overheating, oil leakage, discharge). The set of status reference images marked as normal reflects the image of the power transmission equipment in normal operation, covering different operating conditions and environmental conditions of the equipment. Based on this, in the technical solution of the present application, the real-time status image of the power transmission equipment collected by the inspection drone is obtained, and the set of status reference images marked as normal is extracted from the background database, and the real-time status image and the set of status reference images marked as normal are subjected to image analysis and comparison, so that the status of the equipment can be detected in real time, so as to discover potential problems early and prevent failures from occurring, thereby improving the safety of the equipment.

[0042] In an embodiment of the present application, the state reference feature extraction module 130 is used to extract image features from each state reference image in the set of state reference images marked as being in a normal state to obtain a set of state reference feature maps. Specifically, in an embodiment of the present application, the state reference feature extraction module is used to: pass each state reference image in the set of state reference images marked as being in a normal state through an image feature extractor based on a convolutional neural network model to obtain the set of state reference feature maps. Accordingly, considering that each state reference image in the set of state reference images marked as being in a normal state has implicit feature information indicating that the state is normal under different conditions, and considering that image feature extractors based on convolutional neural networks (CNN) models are widely used in the field of image recognition and analysis and can effectively capture and extract implicit feature information in images. Based on this, in the technical solution of the present application, each state reference image in the set of state reference images marked as being in a normal state is passed through an image feature extractor based on a convolutional neural network model to mine out implicit feature information about the normal state of the power transmission equipment under different conditions from each state reference image, thereby obtaining a set of state reference feature maps.

[0043] In an embodiment of the present application, the feature screening module 140 is used to pass each state reference feature map in the set of state reference feature maps through a feature distiller based on a feature screening network to obtain a set of distilled state reference feature vectors. It should be understood that, considering that each state reference feature map in the set of state reference feature maps reflects the image semantic feature information under normal conditions, these semantic feature information has an important impact on the state reference feature processing. And because each state reference feature map is extracted from each state reference image, it also contains redundant information and noise in the original state reference image, and this information has little impact on the state reference feature processing. Therefore, in order to improve the expressive power of the state reference feature, remove unnecessary information, and retain state reference semantic features that are close to or similar to the state reference feature, in the technical solution of the present application, each state reference feature map in the set of state reference feature maps is passed through a feature distiller based on a feature screening network to obtain a set of distilled state reference feature vectors. It should be understood that feature screening of each state reference feature graph in the set of state reference feature graphs can reduce the influence of state reference features that differ significantly in semantics from the prototype features, while increasing the influence of state reference features that are semantically close to or similar to the prototype features. In particular, expanding each state reference feature graph in the set of state reference feature graphs to obtain a sequence of state reference feature vectors can more meticulously capture and integrate the distilled state reference features in each state reference feature graph, thereby achieving dimensionality reduction of the state reference feature graphs to obtain a more discriminative and representative set of distilled state reference feature vectors, providing better assistance for the subsequent calculation of the semantic similarity representation between the normal state and the real-time state of the power transmission equipment.

[0044] Specifically, in an embodiment of the present application, the feature screening module is configured to: use the feature distiller based on the feature screening network to process each state reference feature graph in the set of state reference feature graphs using the following feature screening formula to obtain the set of distilled state reference feature vectors; wherein the feature screening formula is:

[0045]

[0046]

[0047] Among them, v i (x) and v j (x) are the i-th and j-th state reference feature vectors in the sequence of the state reference feature vectors after the expansion of each state reference feature graph, L is the scale of each state reference feature vector in the sequence of the state reference feature vector, V krepresents the sequence of the state reference feature vectors, M represents the number of state reference feature vectors in the sequence of the state reference feature vectors - 1, D i Represents the feature value of each position in the state reference semantic difference feature vector, N s represents the number of eigenvalues ​​in the state reference semantic difference feature vector, V represents each distilled state reference feature vector in the set of distilled state reference feature vectors, exp(·) represents an exponential function with the natural constant e as the base, and log represents the logarithmic function value with base 2.

[0048] In this embodiment of the present application, the feature joint clustering analysis module 150 is configured to apply a joint clustering analysis network to the set of distilled state reference feature vectors to obtain a clustering feature matrix representing the normal state of the power transmission equipment. Accordingly, considering the potential semantic relationships and patterns between the set of distilled state reference feature vectors, that is, the similarities and correlations between the normal state reference features of different types of power transmission equipment within the set of distilled state reference feature vectors, in order to comprehensively integrate the state reference features within the set of distilled state reference feature vectors and better capture the overall patterns and regularities of the distilled state reference feature vectors, the technical solution of the present application applies a joint clustering analysis network to obtain a clustering feature matrix representing the normal state of the power transmission equipment. That is, by performing cluster analysis on the set of the distillation state reference feature vectors through the joint cluster analysis network, the distillation state reference features in the high-dimensional feature space can be clustered, and the potential data structure and pattern of the distillation state reference can be discovered, so that similar distillation state reference feature vectors can be aggregated together to form a representative and effective cluster characterization feature matrix of the normal state of the transmission equipment, providing a reference for subsequent transmission equipment abnormality detection.

[0049] Figure 3 FIG is a block diagram of a feature joint clustering analysis module in a power transmission and transformation equipment inspection system based on visible light image recognition technology according to an embodiment of the present application. Specifically, in the embodiment of the present application, Figure 3As shown, the feature joint clustering analysis module 150 includes: a matrix construction unit 151, which is used to construct the adjacency matrix and degree matrix of the set of distillation state reference eigenvectors; a Laplace matrix calculation unit 152, which is used to calculate the Laplace matrix based on the adjacency matrix and the degree matrix; a normalization processing unit 153, which is used to normalize the Laplace matrix to obtain a standardized Laplace matrix; an eigenvalue arrangement unit 154, which is used to arrange the eigenvalues ​​of the standardized Laplace matrix from large to small, and extract the first K eigenvalues ​​to calculate the eigenvectors of the first K eigenvalues; and a feature composition unit 155, which is used to standardize the eigenvectors of the first K eigenvalues ​​and compose the eigenvectors of the standardized first K eigenvalues ​​into an eigenvector matrix to obtain the normal state cluster characterization feature matrix of the power transmission equipment.

[0050] More specifically, in an embodiment of the present application, the matrix construction unit is configured to construct an adjacency matrix for the set of distillation state reference eigenvectors, including calculating association weights between respective distillation state reference eigenvectors in the set of distillation state reference eigenvectors using the following weight formula to obtain the adjacency matrix; wherein the weight formula is:

[0051]

[0052] Among them, p i and p j are the i-th and j-th distillation state reference feature vectors in the set of distillation state reference feature vectors, σ is the variance between the i-th distillation state reference feature vector and the j-th distillation state reference feature vector, exp(·) represents an exponential function with the natural constant e as the base, and W i,j is the eigenvalue of the (i, j)th position in the adjacency matrix.

[0053] In an embodiment of the present application, the power transmission device real-time state feature extraction module 160 is configured to extract image features from the real-time state image to obtain a power transmission device real-time state feature matrix. Specifically, in this embodiment, the power transmission device real-time state feature extraction module is configured to pass the real-time state image through the image feature extractor based on the convolutional neural network model to obtain the power transmission device real-time state feature matrix. Similarly, considering that the real-time state image contains implicit feature information of the real-time state of the power transmission device, and convolutional neural networks are deep learning models specifically used for processing and analyzing implicit image features, in the technical solution of the present application, the real-time state image is passed through the image feature extractor based on the convolutional neural network model to obtain the power transmission device real-time state feature matrix. In particular, in one specific embodiment of the present application, the real-time state image is passed through the image feature extractor based on the convolutional neural network model to capture and mine the implicit feature information of the real-time state image regarding the real-time state of the power transmission device to obtain a power transmission device real-time state feature map. The real-time state feature map is then subjected to global mean pooling along the channel dimension to obtain the power transmission device real-time state feature matrix.

[0054] In this embodiment of the present application, the similarity calculation module 170 is configured to calculate the hash similarity between the normal state cluster representation feature matrix of the power transmission device and the real-time state feature matrix of the power transmission device. It should be understood that in order to more accurately compare the semantic similarity between the normal state cluster representation feature matrix and the real-time state feature matrix of the power transmission device, thereby enabling a more precise subsequent determination of whether the power transmission device has a state anomaly, the technical solution of the present application calculates the hash similarity between the normal state cluster representation feature matrix and the real-time state feature matrix of the power transmission device. It is worth noting that hash similarity is a method for quickly comparing the similarity between two data sets, providing a quantitative assessment of the similarity between two feature sets and expressing the degree of similarity between them in numerical form. In other words, by calculating the hash similarity between the normal state cluster representation feature matrix and the real-time state feature matrix of the power transmission device, differences between the real-time state feature matrix and the normal state cluster representation feature matrix can be promptly detected, resulting in a more accurate similarity value and thus improving the accuracy of determining whether the power transmission device has a state anomaly.

[0055] In this embodiment of the present application, the power transmission equipment anomaly determination module 180 is configured to determine whether the power transmission equipment is in an abnormal state based on a comparison between the hash similarity and a predetermined threshold. Specifically, the module automatically determines whether the power transmission equipment is in an abnormal state by comparing the hash similarity between the normal state clustering feature matrix of the power transmission equipment and the real-time state feature matrix of the power transmission equipment with a predetermined threshold. This approach enables real-time monitoring of the power transmission equipment status and accurately determines whether the power transmission equipment is in an abnormal state, reducing the likelihood of misjudgments and missed detections. This reduces manual inspection costs, improves inspection efficiency, and enhances the safety and reliability of power grid operations.

[0056] It is worth mentioning that those skilled in the art should be aware that before applying a deep neural network model for inference, the deep neural network model must first be trained so that the deep neural network can implement specific functional capabilities.

[0057] Specifically, in the technical solution of the present application, the power transmission and transformation equipment inspection system based on visible light image recognition technology also includes a training module for training the image feature extractor based on the convolutional neural network model, the feature distiller based on the feature screening network, and the joint clustering analysis network.

[0058] Figure 4 FIG is a block diagram of a training module in a power transmission and transformation equipment inspection system based on visible light image recognition technology according to an embodiment of the present application. Specifically, in the embodiment of the present application, Figure 4As shown, the training module 200 includes: a training data acquisition unit 210 for acquiring training data, wherein the training data includes real-time state images of the power transmission equipment collected by the inspection drone, a set of training state reference images marked as normal, and true hash similarity extracted from the background database; a training state reference feature extraction unit 220 for passing each training state reference image in the set of training state reference images marked as normal through the image feature extractor based on the convolutional neural network model to obtain a set of training state reference feature maps; a training feature distillation unit 230 for passing each training state reference feature map in the set of training state reference feature maps through the feature distillation based on the feature screening network to obtain a set of training distilled state reference feature vectors; a training power transmission equipment normal state cluster representation unit 240 for passing the set of training distilled state reference feature vectors through the joint cluster analysis network to obtain a training power transmission equipment normal state cluster representation feature matrix; a training power transmission equipment real-time state extraction unit 250 for passing the training real-time state images through the convolutional neural network model based on the image feature extractor based on the convolutional neural network model to obtain a set of training distilled state reference feature maps; An image feature extractor is used to obtain a training power transmission device real-time state feature matrix; a feature optimization unit 260 is used to optimize the training power transmission device normal state cluster representation feature matrix and the training power transmission device real-time state feature matrix to obtain an optimized training power transmission device normal state cluster representation feature matrix and an optimized training power transmission device real-time state feature matrix; a predicted hash similarity calculation unit 270 is used to calculate the predicted hash similarity between the optimized training power transmission device normal state cluster representation feature matrix and the optimized training power transmission device real-time state feature matrix; a difference loss function value calculation unit 280 is used to calculate the variance between the predicted hash similarity and the true hash similarity as a difference loss function value; and a training unit 290 is used to train the image feature extractor based on the convolutional neural network model, the feature distiller based on the feature screening network, and the joint cluster analysis network based on the difference loss function value and through gradient descent back propagation, wherein the training power transmission device normal state cluster representation feature matrix and the training power transmission device real-time state feature matrix are optimized in each model training iteration.

[0059] Specifically, the training unit 290 is used to train the image feature extractor based on the convolutional neural network model, the feature distiller based on the feature screening network, and the joint clustering analysis network based on the difference loss function value and through back propagation of gradient descent, wherein the training power transmission device normal state clustering representation feature matrix and the training power transmission device real-time state feature matrix are optimized at each training iteration of the model. It should be understood that in the technical solution of the present application, each training state reference feature map in the set of training state reference feature maps expresses the image semantic features of each training state reference image in the set of training state reference images, and the training power transmission device real-time state feature matrix expresses the image semantic features of the training real-time state image, and considering the image semantic differences of the source images and the image semantic feature set screening and joint clustering operations after each training state reference feature map in the set of training state reference feature maps passes through the feature distiller based on the feature screening network and the joint clustering analysis network, the training power transmission device normal state clustering representation feature matrix is ​​optimized. There is a significant difference in feature space distribution between the image semantic feature space distribution and the image semantic feature space distribution of the training power transmission device real-time state feature matrix. This makes it difficult to regress the image semantic feature distribution when calculating the hash similarity between the training power transmission device normal state cluster representation feature matrix and the training power transmission device real-time state feature matrix during training. In other words, this affects the similarity regression effect between the image semantic feature space distribution of the training power transmission device normal state cluster representation feature matrix and the training power transmission device real-time state feature matrix based on the overall numerical distribution of the eigenvalues ​​of each feature matrix, thus affecting the training speed and the accuracy of the training results. Based on this, in the technical solution of the present application, the training power transmission device normal state cluster representation feature matrix and the training power transmission device real-time state feature matrix are optimized during each training iteration of the model.

[0060] Specifically, in an embodiment of the present application, the training unit includes: a probability coefficient calculation subunit, which is used to perform scenario-based quasi-probability logic reasoning on the training power transmission device normal state cluster characterization feature matrix and the training power transmission device real-time state feature matrix to obtain corresponding probability coefficients; and a weighted optimization subunit, which is used to perform weighted optimization on the training power transmission device normal state cluster characterization feature matrix and the training power transmission device real-time state feature matrix using the corresponding probability coefficients as weighting coefficients to obtain the optimized training power transmission device normal state cluster characterization feature matrix and the optimized training power transmission device real-time state feature matrix.

[0061] More specifically, in an embodiment of the present application, the probability coefficient calculation subunit is configured to perform scenario-based quasi-probabilistic logical reasoning on the training power transmission device normal state cluster characterization feature matrix and the training power transmission device real-time state feature matrix using the following optimization formula to obtain the corresponding probability coefficients; wherein the optimization formula is:

[0062]

[0063] Among them, m i,j is the eigenvalue of the (i, j)th position of the cluster characterization feature matrix of the normal state of the training power transmission equipment and the real-time state feature matrix of the training power transmission equipment, φ(m i,j ) represents the probabilistic function of the eigenvalue, that is, the eigenvalue m i,j A probabilistic function mapped to the interval [0,1], S is the scale of the normal state clustering representation feature matrix of the training power transmission device and the real-time state feature matrix of the training power transmission device, that is, the width multiplied by the height, p is the hash similarity value, and α is a weight hyperparameter, ω is the probability coefficient corresponding to the normal state clustering representation feature matrix of the training power transmission device and the real-time state feature matrix of the training power transmission device.

[0064] Specifically, for the characteristic scenarios corresponding to the training power transmission device normal state cluster representation feature matrix and the training power transmission device real-time state feature matrix, scenario saturation regression probabilistic reasoning logical association is adopted through probability distribution prospect constraints and relative probability mapping response assumptions, thereby imparting scenario concept ontological cognition to the feature sets of the training power transmission device normal state cluster representation feature matrix and the training power transmission device real-time state feature matrix. In other words, the overall distribution is internally aligned with the scenario-based regression probabilistic logical reasoning under the regression process, thereby improving the understanding of the scenario distribution of the training power transmission device normal state cluster representation feature matrix and the training power transmission device real-time state feature matrix for similarity regression cognition. Thus, by performing weighted optimization on the training power transmission device normal state cluster representation feature matrix and the training power transmission device real-time state feature matrix using the coefficient ω, the regression effect of the hash similarity calculation between the training power transmission device normal state cluster representation feature matrix and the training power transmission device real-time state feature matrix can be improved, thereby improving the training speed and the accuracy of the training results. In this way, real-time monitoring of the status of power transmission equipment can be achieved, and whether the power transmission equipment has abnormal status can be accurately judged, reducing the possibility of misjudgment and missed detection. At the same time, it reduces the cost of manual inspections, improves inspection efficiency, and enhances the safety and reliability of power grid operation.

[0065] In summary, the power transmission and transformation equipment inspection system 100 based on visible light image recognition technology according to the embodiment of the present application is explained. It uses the real-time status image of the power transmission equipment collected by the inspection drone, and extracts a set of state reference images marked as normal from the background database, and uses the image recognition and analysis algorithm based on deep learning to perform image processing and analysis on the real-time status image and the state reference image marked as normal, so as to automatically determine whether the power transmission equipment has an abnormal state based on the comparison between the semantic similarity between the real-time status image and the state reference image marked as normal and a predetermined threshold. In this way, real-time monitoring of the status of the power transmission equipment can be achieved, and whether the power transmission equipment has an abnormal state can be accurately determined, reducing the possibility of misjudgment and missed detection, while reducing the cost of manual inspection, improving inspection efficiency, and enhancing the safety and reliability of power grid operation.

[0066] As described above, the power transmission and transformation equipment inspection system 100 based on visible light image recognition technology according to the embodiment of the present application can be implemented in various wireless terminals, such as a server having a power transmission and transformation equipment inspection algorithm based on visible light image recognition technology. In one possible implementation, the power transmission and transformation equipment inspection system 100 based on visible light image recognition technology according to the embodiment of the present application can be integrated into the wireless terminal as a software module and / or hardware module. For example, the power transmission and transformation equipment inspection system 100 based on visible light image recognition technology can be a software module in the operating system of the wireless terminal, or it can be an application developed for the wireless terminal; of course, the power transmission and transformation equipment inspection system 100 based on visible light image recognition technology can also be one of the many hardware modules of the wireless terminal.

[0067] Alternatively, in another example, the power transmission and transformation equipment inspection system 100 based on visible light image recognition technology and the wireless terminal may also be separate devices, and the power transmission and transformation equipment inspection system 100 based on visible light image recognition technology may be connected to the wireless terminal through a wired and / or wireless network and transmit interactive information in accordance with an agreed data format.

[0068] Figure 5 Flowchart of the power transmission and transformation equipment inspection method based on visible light image recognition technology according to an embodiment of the present application. Figure 5As shown, according to the embodiment of the present application, the power transmission and transformation equipment inspection method based on visible light image recognition technology includes: S110, obtaining a real-time status image of the power transmission equipment collected by the inspection drone; S120, extracting a set of state reference images marked as normal from the background database; S130, performing image feature extraction on each state reference image in the set of state reference images marked as normal to obtain a set of state reference feature graphs; S140, performing a feature distiller based on a feature screening network on each state reference feature graph in the set of state reference feature graphs to obtain a set of state reference feature graphs. Obtain a set of distilled state reference feature vectors; S150, apply the set of distilled state reference feature vectors to a joint cluster analysis network to obtain a normal state cluster representation feature matrix of the power transmission device; S160, perform image feature extraction on the real-time state image to obtain a real-time state feature matrix of the power transmission device; S170, calculate a hash similarity between the normal state cluster representation feature matrix of the power transmission device and the real-time state feature matrix of the power transmission device; and S180, determine whether the power transmission device has an abnormal state based on a comparison between the hash similarity and a predetermined threshold.

[0069] Here, those skilled in the art will appreciate that the specific operations of each step in the above-mentioned power transmission and transformation equipment inspection method based on visible light image recognition technology have been referred to above. Figures 1 to 4 The invention has been introduced in detail in the description of the power transmission and transformation equipment inspection system based on visible light image recognition technology, and therefore, its repeated description will be omitted.

[0070] While various implementations of the present disclosure have been described above, the foregoing description is intended to be illustrative and not exhaustive. The disclosure is not limited to the disclosed implementations, and numerous modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described implementations. The terminology used herein is selected to best explain the principles of the implementations, their practical applications, or improvements to existing technologies, or to enable others skilled in the art to understand the various implementations disclosed herein.

Claims

1. A power transmission and transformation equipment inspection system based on visible light image recognition technology, characterized in that: include: The real-time status acquisition module of the power transmission equipment is used to obtain the real-time status images of the power transmission equipment collected by the inspection drone; A state reference image acquisition module is used to extract a set of state reference images marked as normal from a background database; a state reference feature extraction module, configured to extract image features from each state reference image in the set of state reference images marked as being in a normal state to obtain a set of state reference feature maps; a feature screening module, configured to pass each state reference feature graph in the set of state reference feature graphs through a feature distiller based on a feature screening network to obtain a set of distilled state reference feature vectors; A feature joint clustering analysis module, configured to pass the set of distillation state reference feature vectors through a joint clustering analysis network to obtain a cluster characterization feature matrix of a normal state of the power transmission equipment; The feature joint clustering analysis module includes: a matrix construction unit, configured to construct an adjacency matrix and a degree matrix of the set of distillation state reference eigenvectors; a Laplace matrix calculation unit, configured to calculate a Laplace matrix based on the adjacency matrix and the degree matrix; a normalization processing unit, configured to perform normalization processing on the Laplace matrix to obtain a normalized Laplace matrix; an eigenvalue arrangement unit, configured to arrange the eigenvalues ​​of the normalized Laplace matrix from largest to smallest, and extract the first K eigenvalues ​​to calculate the eigenvectors of the first K eigenvalues; a feature composition unit, configured to normalize the eigenvectors of the first K eigenvalues ​​and compose the standardized eigenvectors of the first K eigenvalues ​​into a eigenvector matrix to obtain a cluster characterization feature matrix of the normal state of the power transmission equipment; A power transmission equipment real-time state feature extraction module, configured to extract image features from the real-time state image to obtain a power transmission equipment real-time state feature matrix; A similarity calculation module, configured to calculate a hash similarity between the normal state cluster representation feature matrix of the power transmission device and the real-time state feature matrix of the power transmission device; a power transmission device abnormality judgment module, configured to determine whether the power transmission device has an abnormal state based on a comparison between the hash similarity and a predetermined threshold; The feature screening module is configured to: use the feature distillation based on the feature screening network to process each state reference feature graph in the set of state reference feature graphs using the following feature screening formula to obtain the set of distilled state reference feature vectors; Wherein, the feature screening formula is: Among them, v i (x) and v j (x) are the i-th and j-th state reference feature vectors in the sequence of the state reference feature vectors after the expansion of each state reference feature graph, L is the scale of each state reference feature vector in the sequence of the state reference feature vector, V k represents the sequence of the state reference feature vectors, M represents the number of state reference feature vectors in the sequence of the state reference feature vectors - 1, D i Represents the feature value of each position in the state reference semantic difference feature vector, N s represents the number of eigenvalues ​​in the state reference semantic difference feature vector, V represents each distilled state reference feature vector in the set of distilled state reference feature vectors, exp(·) represents an exponential function with the natural constant e as the base, and log represents the logarithmic function value with base 2.

2. The power transmission and transformation equipment inspection system based on visible light image recognition technology according to claim 1 is characterized in that: The state reference feature extraction module is used to: pass each state reference image in the set of state reference images marked as normal through an image feature extractor based on a convolutional neural network model to obtain the set of state reference feature maps.

3. The power transmission and transformation equipment inspection system based on visible light image recognition technology according to claim 2 is characterized in that: The matrix construction unit is configured to construct an adjacency matrix of the set of distillation state reference eigenvectors, comprising: calculating association weight values ​​between the respective distillation state reference eigenvectors in the set of distillation state reference eigenvectors using the following weight formula to obtain the adjacency matrix; Wherein, the weight formula is: Among them, p i and p j are the i-th and j-th distillation state reference feature vectors in the set of distillation state reference feature vectors, σ is the variance between the i-th distillation state reference feature vector and the j-th distillation state reference feature vector, exp(·) represents an exponential function with the natural constant e as the base, and W i,j is the eigenvalue of the (i, j)th position in the adjacency matrix.

4. The power transmission and transformation equipment inspection system based on visible light image recognition technology according to claim 3 is characterized in that: The power transmission equipment real-time state feature extraction module is used to: pass the real-time state image through the image feature extractor based on the convolutional neural network model to obtain the real-time state feature matrix of the power transmission equipment.

5. The power transmission and transformation equipment inspection system based on visible light image recognition technology according to claim 4 is characterized in that: It also includes a training module for training the image feature extractor based on the convolutional neural network model, the feature distiller based on the feature screening network and the joint clustering analysis network.

6. The power transmission and transformation equipment inspection system based on visible light image recognition technology according to claim 5 is characterized in that: The training module includes: A training data acquisition unit is configured to acquire training data, wherein the training data includes real-time state images of the power transmission equipment collected by the inspection drone, a set of training state reference images labeled as normal extracted from a background database, and true hash similarity; a training state reference feature extraction unit, configured to pass each training state reference image in the set of training state reference images marked as being in a normal state through the image feature extractor based on the convolutional neural network model to obtain a set of training state reference feature maps; a training feature distillation unit, configured to pass each training state reference feature map in the set of training state reference feature maps through the feature distiller based on the feature screening network to obtain a set of training distillation state reference feature vectors; A training power transmission equipment normal state cluster characterization unit, configured to pass the set of training distilled state reference feature vectors through the joint cluster analysis network to obtain a training power transmission equipment normal state cluster characterization feature matrix; A training power transmission device real-time state extraction unit, configured to pass the training real-time state image through the image feature extractor based on the convolutional neural network model to obtain a training power transmission device real-time state feature matrix; a feature optimization unit, configured to optimize the normal-state clustering characterization feature matrix of the training power transmission equipment and the real-time state feature matrix of the training power transmission equipment to obtain an optimized normal-state clustering characterization feature matrix of the training power transmission equipment and an optimized real-time state feature matrix of the training power transmission equipment; A predicted hash similarity calculation unit, configured to calculate a predicted hash similarity between the cluster characterization feature matrix of the optimized training power transmission device in a normal state and the feature matrix of the optimized training power transmission device in a real-time state; a difference loss function value calculation unit, configured to calculate a variance between the predicted hash similarity and the true hash similarity as a difference loss function value; A training unit is used to train the image feature extractor based on the convolutional neural network model, the feature distiller based on the feature screening network, and the joint clustering analysis network based on the difference loss function value and through back propagation of gradient descent, wherein, during each training iteration of the model, the normal state cluster characterization feature matrix of the training power transmission equipment and the real-time state feature matrix of the training power transmission equipment are optimized.

7. The power transmission and transformation equipment inspection system based on visible light image recognition technology according to claim 6 is characterized in that: The feature optimization unit includes: A probability coefficient calculation subunit, configured to perform scenario-based quasi-probability logic reasoning on the normal state cluster characterization feature matrix of the training power transmission equipment and the real-time state feature matrix of the training power transmission equipment to obtain corresponding probability coefficients; The weighted optimization subunit is used to perform weighted optimization on the training power transmission equipment normal state cluster characterization feature matrix and the training power transmission equipment real-time state feature matrix respectively using the corresponding probability coefficient as a weighting coefficient to obtain the optimized training power transmission equipment normal state cluster characterization feature matrix and the optimized training power transmission equipment real-time state feature matrix.

8. A method for inspecting power transmission and transformation equipment based on visible light image recognition technology, characterized in that: include: Obtain real-time status images of power transmission equipment collected by inspection drones; Extracting a set of reference images marked as being in a normal state from a background database; Performing image feature extraction on each state reference image in the set of state reference images marked as being in a normal state to obtain a set of state reference feature maps; Passing each state reference feature graph in the set of state reference feature graphs through a feature distiller based on a feature screening network to obtain a set of distilled state reference feature vectors; Passing the set of distillation state reference feature vectors through a joint cluster analysis network to obtain a normal state cluster characterization feature matrix of the power transmission equipment; The joint cluster analysis network includes: for constructing an adjacency matrix and a degree matrix of a set of reference feature vectors of the distilled state; For calculating a Laplacian matrix based on the adjacency matrix and the degree matrix; Used to perform normalization processing on the Laplace matrix to obtain a normalized Laplace matrix; Used to arrange the eigenvalues ​​of the normalized Laplace matrix from large to small, and extract the first K eigenvalues ​​to calculate the eigenvectors of the first K eigenvalues; for normalizing the eigenvectors of the first K eigenvalues ​​and forming the eigenvectors of the first K eigenvalues ​​after normalization into an eigenvector matrix to obtain a clustering characterization feature matrix of the normal state of the power transmission equipment; Performing image feature extraction on the real-time state image to obtain a real-time state feature matrix of the power transmission equipment; Calculating the hash similarity between the normal state cluster representation feature matrix of the power transmission device and the real-time state feature matrix of the power transmission device; determining whether the power transmission device has an abnormal state based on a comparison between the hash similarity and a predetermined threshold; wherein, using the feature distiller based on the feature screening network to process each state reference feature graph in the set of state reference feature graphs using the following feature screening formula to obtain the set of distilled state reference feature vectors; Wherein, the feature screening formula is: Among them, v i (x) and v j (x) are the i-th and j-th state reference feature vectors in the sequence of the state reference feature vectors after the expansion of each state reference feature graph, L is the scale of each state reference feature vector in the sequence of the state reference feature vector, V k represents the sequence of the state reference feature vectors, M represents the number of state reference feature vectors in the sequence of the state reference feature vectors - 1, D i Represents the feature value of each position in the state reference semantic difference feature vector, N s represents the number of eigenvalues ​​in the state reference semantic difference feature vector, V represents each distilled state reference feature vector in the set of distilled state reference feature vectors, exp(·) represents an exponential function with the natural constant e as the base, and log represents the logarithmic function value with base 2.

Images