Three-dimensional visual electric power inspection method

Abstract

The invention provides a three-dimensional visual electric power inspection method, and relates to the technical field of electric power equipment inspection, and the method comprises the steps: collecting real-time operation data and historical operation data, building an inspection model, and training the inspection model through the historical operation data, and inputting the real-time operation data into the trained inspection model, and outputting a first inspection result. According to the invention, the inspection model is established by using the deep learning algorithm, and the historical operation data is deeply analyzed and processed through the inspection model, so that the capability of judging the real-time operation data is obtained, and the real-time monitoring and accurate detection of the inspection target are realized.

Description

Technical Field

[0001] The present application relates to the technical field of power equipment inspection, and in particular to a three-dimensional visualized power inspection method. Background Art

[0002] In the field of power inspection, the traditional inspection method mainly relies on manual on-site inspection, which is not only time-consuming and labor-intensive, but also limited by the experience and judgment of the inspectors, and may lead to problems such as missed inspections and misjudgments. With the increasing complexity and scale of power systems, traditional inspection methods can no longer meet the management needs of modern power systems.

[0003] In recent years, with the continuous development of information technology, especially the widespread application of technologies such as big data, cloud computing and artificial intelligence, new solutions have been provided for power inspection. However, the data in power inspection includes real-time operation data, historical operation data, etc. These data are diverse and complex, and are often affected by many factors, including environmental factors, equipment aging, load changes, etc. The interaction between these influencing factors makes the operation status of the power system very complicated, and traditional inspection methods are difficult to fully capture and analyze these changes. Summary of the invention

[0004] In order to accurately capture and analyze the changing patterns of data in the power system, the present application provides a three-dimensional visualized power inspection method.

[0005] This application provides a three-dimensional visualized power inspection method, which adopts the following technical solutions: A three-dimensional visual power inspection method includes the following steps: Data collection: including the first collection and the second collection; First collection: collect real-time operation data of inspection targets; Second collection: collect historical operation data of inspection targets; First modeling: Based on the deep learning algorithm, an inspection model is established, and the inspection model includes: a CNN sub-model, a BI-LSTM sub-model and a decision tree sub-model; Model training: including feature extraction, first training and second training to obtain the trained inspection model; Feature extraction: Use CNN sub-model to extract n running features; First training: input n running features into the BI-LSTM sub-model to obtain the output data of the BI-LSTM sub-model; Second training: input n running features and the output data of the BI-LSTM sub-model into the decision tree sub-model and output the decision results; Detection: Input the real-time operation data into the trained inspection model and output the first inspection result.

[0006] The present application first collects real-time operation data and historical operation data. The current working status of the inspection target can be obtained through the real-time operation data, and the past working status of the inspection target can be obtained through the historical operation data, which is helpful to identify potential problems or trends. Then, an inspection model is established, and the inspection model includes: a CNN sub-model, a BI-LSTM sub-model and a decision tree sub-model. The CNN sub-model is good at extracting local features from the input data and can accurately extract the operation features in the historical operation data of the inspection target. The BI-LSTM sub-model can process sequence data (i.e., n operation features), capture the time dependency in the data, and help analyze the trend of the inspection target changing over time. The decision tree sub-model can make decisions based on the extracted features and the output of the BI-LSTM, and output the decision results. Then, the inspection model is trained with historical operation data to obtain a trained inspection model, and then the real-time operation data is input into the trained inspection model to output the first inspection result. The present application combines real-time and historical data, and through multiple sub-models to collaboratively process data, it can comprehensively and accurately analyze the operation data of the inspection target.

[0007] Optionally, the step of data collection further includes: a third collection: collecting real-time images and historical images of the inspection target, wherein the historical images include normal images, abnormal images, and defect labels corresponding to the abnormal images; and after executing the step of data collection and before executing the step of first modeling, further including: First segmentation: using an image segmentation algorithm to segment the real-time image to obtain a plurality of first segmentation blocks; First judgment: judging whether there is occlusion in the inspection target based on the first segmentation block, if so, executing the first calculation step; if not, executing the first modeling step; First calculation: using the cosine similarity algorithm to calculate the similarity between the real-time operation data and the historical operation data, and taking the historical operation data when the similarity with the real-time operation data is the maximum as the first data; In the detection step, the first data is used to replace the real-time operation data and input into the inspection model.

[0008] The present application divides the real-time image into a plurality of first segments by an image segmentation algorithm, and determines whether there is occlusion in the inspection target by analyzing the first segments. If there is occlusion, the historical operation data most similar to the real-time operation data is selected as the data to be processed later by calculating the similarity between the real-time operation data and the historical operation data. The real-time operation data under occlusion is incomplete, so it cannot accurately reflect the real state of the inspection target, which is easy to cause false alarms or omissions. Using historical operation data instead can reduce the risk of such false alarms and omissions based on the processing results of similar situations in the past. Therefore, the present application uses historical operation data to enhance the recognition ability of the inspection model, especially when the inspection target in the real-time image is occluded, the inspection result of the inspection target can still be inferred by analyzing the historical operation data. In the detection step, the historical operation data with the greatest similarity to the real-time operation data is used instead of the real-time operation data to be input into the inspection model, which helps to reduce the misjudgment caused by the occlusion of the inspection target and improve the stability and accuracy of the system.

[0009] Optionally, after performing the first calculation step and before performing the first modeling step, the method further includes: Retraining: Use historical images to train the inspection model again to obtain a retrained inspection model; First acquisition: acquiring all historical images corresponding to the first data, and recording one historical image as a first image; Second calculation: using a hash algorithm to calculate the similarity between the real-time image and each first image, and selecting the first image when the similarity with the real-time image is the maximum as the second image; In the detection step, the second image and the real-time operation data are input into the retrained inspection model.

[0010] The present application uses image data to retrain the inspection model to obtain a retrained inspection model, which can use image-assisted operation data to realize inspection of power equipment. The present application also obtains historical data with the greatest similarity to real-time operation data, and obtains its corresponding historical images. The similarity between the real-time image and these historical images is then calculated through a hash algorithm, the most similar historical image is selected as the second image, and the second image and the real-time operation data are input into the retrained inspection model, that is, the unobstructed historical image is used instead of the obstructed real-time image to improve the accuracy of the detection results.

[0011] Optionally, after performing the second calculation step and before performing the first modeling step, the method further includes: Second segmentation: segmenting the second image using an image segmentation algorithm to obtain a second segmentation block containing the inspection target; First fusion: performing image fusion processing on the second segmented block containing the inspection target and the real-time image to obtain a fused image; In the detection step, the fused image and real-time operation data are input into the retrained inspection model.

[0012] The present application uses an image segmentation algorithm to segment the second image (i.e., the historical image that is most similar to the real-time image) in order to identify and extract the second segmented block containing the inspection target. The second segmented block containing the inspection target is subjected to image fusion processing with the real-time image, and the corresponding obscured or blurred area in the real-time image can be replaced with the second segmented block (i.e., the inspection target area that is clearer or unobstructed extracted from the historical image), thereby improving the image quality input into the inspection model. In particular, when there are obstructed areas in the real-time image, the present application can use the information in the historical image to make up for these deficiencies, thereby more accurately judging the state of the inspection target. Through the two steps of the second segmentation and the first fusion, the present application can use the clear or unobstructed areas in the historical image to optimize the real-time image, thereby improving the image quality input into the inspection model. Since the image quality input into the inspection model is improved, the recognition accuracy of the present application will also be improved accordingly.

[0013] Optionally, after executing the model training step and before executing the detection step, the method further includes: The fifth process: modifying the pixel value of each pixel point in the segmented block that does not include the inspection target to 0; Second fusion: fusing the segmented blocks including the inspection target and the segmented blocks after the pixel values ​​are modified to obtain a fused real-time image, and using the fused real-time image as a new fused image.

[0014] The present application modifies the pixel value of each pixel point in the segmentation block that does not include the inspection target to 0, so that the detection algorithm can focus more on the inspection target itself, thereby removing background information irrelevant to the inspection target, reducing noise and interference in subsequent detection steps, and helping to improve the accuracy and efficiency of detection. The present application performs fusion processing on the segmentation block including the inspection target and the segmentation block after the pixel value is modified, that is, the processed segmentation block is recombined into a complete real-time image, which can retain important information of the inspection target while removing the background information. Since the background information is removed, the amount of information in the fused real-time image is reduced, and the amount of calculation and time required for the detection algorithm during processing will also be reduced accordingly, thereby improving the efficiency of detection.

[0015] Optionally, after the first acquisition step and before the second calculation step, the method further includes: First integration: integrating the first image and the historical operation data corresponding to the first image into a training set; Second modeling: establish a generative adversarial network model, use the training set to train the generative adversarial network model, and obtain the trained generative adversarial network model; First prediction: input the real-time running data into the trained generative adversarial network model and output multiple predicted images; Add occlusion blocks: randomly add occlusion blocks to each predicted image, and use the predicted image after adding the occlusion blocks as the new predicted image; Third calculation: respectively calculating the similarity between the real-time image and each new predicted image, taking the new predicted image corresponding to the maximum similarity with the real-time image as the third image, and performing the detection step; In the detection step, the third image is used to replace the second image and input into the retrained inspection model to output the second inspection result.

[0016] This application integrates the first image and the corresponding historical operation data into a training set, then establishes a GAN model and trains it using the training set. The GAN model can learn the distribution of the operation data and generate a predicted image similar to the real image through adversarial training of the generator and the discriminator. Then, the real-time operation data is input into the trained GAN model, and multiple predicted images are output. Then, mask blocks are randomly added to each predicted image to simulate the occlusion of the training target. Then, the similarity between the real-time image and each predicted image after adding the mask block is calculated, and the predicted image with the greatest similarity to the real-time image is selected as the third image for subsequent detection steps. This application compares the similarity between the real-time image and the predicted image, selects the image that best matches the current inspection scene for subsequent detection, and in the detection step, uses the third image instead of the second image to input into the inspection model, outputs the second inspection result, and enhances the judgment ability of the inspection model by using the predicted image generated by the GAN model, especially when dealing with occlusion or complex scenes, the effect is more significant. Therefore, this application generates predicted images through the GAN model and simulates occlusion situations, which can better handle occlusion problems in images and improve the robustness of the inspection model. The GAN model can generate a variety of possible images based on real-time operation data, and can provide more training samples and test scenarios to enhance generalization capabilities. This application selects the predicted image that best matches the current inspection scenario for subsequent processing, and can output more accurate and reliable inspection results.

[0017] Optionally, after the step of performing the detection, the method further includes: Set labels: Use target detection algorithms to add corresponding labels to new predicted images; Second judgment: judge whether the label in the step of setting the label is the same as the second inspection result. If so, execute the step of data collection; if not, execute the step of first prediction.

[0018] This application first uses a target detection algorithm to add a corresponding label to the new predicted image, and then by judging whether the label in the step of setting the label is the same as the second inspection result, it can check whether the image predicted by the GAN model is accurate. If the label is the same as the second inspection result, it means that the image predicted by the GAN model is very accurate. Then, the data collection step is performed to start a new round of inspection process. If the label is different from the second inspection result, it means that the image predicted by the GAN model is not accurate enough. Then, the first prediction step is performed, the predicted image is regenerated, and the detection process is re-entered. This application can flexibly adjust its processing flow according to the matching of the detection result and the label. If the detection result is correct, the inspection continues; if the detection result is incorrect, the first prediction step is executed for correction, which reflects the adaptability and flexibility of this application.

[0019] Optionally, the data collection step further includes: fourth collection: collecting three-dimensional point cloud data and real-time environmental data of the inspection target; and after the step of performing detection, further including: Third modeling: Use 3D point cloud data to create a 3D model of the inspection target; First processing: deploying the real-time operation data and the real-time environment data into the three-dimensional model to obtain a fused three-dimensional model; Run simulation: Use simulation software to run simulation on the fused 3D model to obtain simulation results; Third judgment: judging whether the simulation result is consistent with the first inspection result, if so, executing the output step; if not, executing the feedback step; Output: Output simulation results and the first inspection results; Feedback: Feedback the simulation results and the first inspection results to the operation and maintenance personnel.

[0020] This application collects the three-dimensional point cloud data and real-time environmental data of the inspection target, among which the three-dimensional point cloud data provides the spatial shape and location information of the inspection target, while the real-time environmental data reflects the environmental conditions of the inspection target. Then, the three-dimensional point cloud data is used to create a three-dimensional model of the inspection target, and the inspection target is promoted from two-dimensional image data to three-dimensional space, which can more intuitively understand and analyze the inspection target. Then, the real-time operation data and real-time environmental data are deployed into the three-dimensional model to obtain a fused three-dimensional model. This step integrates the dynamic information and environmental information of the inspection target into the three-dimensional model, making the three-dimensional model closer to the actual situation. Then, the simulation software is used to run the fused three-dimensional model to obtain the simulation results, so as to simulate the operation status of the inspection target and predict its state at the next moment, so as to provide decision support for the operation and maintenance personnel.

[0021] Optionally, after executing the step of performing the first processing and before executing the step of running the simulation, the method further includes: Second extraction: extracting feature points of the inspection target in the real-time image and marking corresponding points of the feature points in the three-dimensional model; Second processing: using computer vision technology to align the real-time image with the fused 3D model to obtain an aligned 3D model; In the step of running the simulation, the aligned three-dimensional model is used instead of the fused three-dimensional model to run the simulation.

[0022] The present application extracts the characteristic points of the inspection target in the real-time image, and also marks the corresponding points of these characteristic points in the three-dimensional model, so that the present application can establish the corresponding relationship between the real-time image and the three-dimensional model by extracting and matching the characteristic points, thereby improving the recognition accuracy of the inspection target. The present application uses computer vision technology to align the real-time image with the fused three-dimensional model, and obtains the aligned three-dimensional model. The position and posture of the three-dimensional model can be adjusted to make it consistent with the inspection target in the real-time image, thereby improving the matching degree between the three-dimensional model and the actual situation. In the step of running the simulation, the aligned three-dimensional model is used instead of the fused three-dimensional model to run the simulation, so that the simulation results are closer to the actual situation, which helps to reduce the difference between the simulation results and the actual situation, improve the accuracy of the simulation, and also help the operation and maintenance personnel to more accurately understand the status of the inspection target and take corresponding measures to deal with it.

[0023] Optionally, after executing the third judgment step and before executing the output step, the method further includes: Planning paths: According to the location of the inspection target, set m inspection paths; Fifth collection: collect the length, time and energy consumption of each inspection path, and integrate the length, time and energy consumption of each inspection path into a vector, which is recorded as the first vector; The third process: concatenate the m first vectors into a first matrix, perform zero mean processing on the rows or columns of the first matrix, and obtain a second matrix; The third calculation: calculate the eigenvalue and eigenvector of the second matrix, and record the eigenvector corresponding to the maximum eigenvalue as the second vector; Fourth process: traverse each element in the second vector, and obtain the element with the largest value, which is recorded as the second data; Set path: take the inspection path corresponding to the second data as the optimal inspection path; The outputting step also includes outputting an optimal inspection path.

[0024] This application first sets m inspection paths according to the location of the inspection target, then collects the length, time and energy consumption of each inspection path, and integrates this information into a vector, recorded as the first vector. The first vector contains the key performance indicators of the inspection path, which provides data support for subsequent data processing. Then, the m first vectors are spliced ​​into a first matrix, and the rows or columns of the first matrix are processed with zero mean, that is, the same type of data (such as length) is standardized, and the processed first matrix is ​​recorded as the second matrix. After that, the eigenvalues ​​and eigenvectors of the second matrix are calculated, and the eigenvector corresponding to the maximum eigenvalue is recorded as the second vector. This application can more comprehensively understand the advantages and disadvantages of different inspection paths by planning paths and collecting key performance indicators, so as to quickly determine the optimal inspection path. After that, each element in the second vector is traversed, and the element with the largest value is obtained, which is recorded as the second data. By finding the maximum value in the eigenvector, this application determines the candidate path of the optimal inspection path. After that, the inspection path corresponding to the second data is used as the optimal inspection path. Later, in the output step, the content of outputting the optimal inspection path was added, so that the operation and maintenance personnel can intuitively understand the information of the optimal inspection path, which helps to improve the inspection efficiency and accuracy.

[0025] In summary, the present application includes at least one of the following beneficial technical effects: 1. This application uses an inspection model to conduct in-depth analysis and processing of operating data, which can accurately capture and analyze the changing patterns of data in the power system and help analyze the trend of inspection targets changing over time.

[0026] 2. This application collects three-dimensional point cloud data and real-time environmental data of the inspection target, among which the three-dimensional point cloud data provides the spatial shape and position information of the inspection target, while the real-time environmental data reflects the environmental conditions of the inspection target. The three-dimensional point cloud data is then used to create a three-dimensional model of the inspection target, which promotes the inspection target from two-dimensional image data to three-dimensional space, so that the inspection target can be understood and analyzed more intuitively.

[0027] 3. This application first sets m inspection paths according to the location of the inspection target, and then collects the length, time and energy consumption of each inspection path, and integrates this information into a vector, recorded as the first vector. The first vector contains the key performance indicators of the inspection path, which provides data support for subsequent data processing. Then, the m first vectors are spliced ​​into a first matrix, and the rows or columns of the first matrix are processed with zero mean, that is, the same type of data (such as length) is standardized, and the processed first matrix is ​​recorded as the second matrix. Then, the eigenvalues ​​and eigenvectors of the second matrix are calculated, and the eigenvector corresponding to the maximum eigenvalue is recorded as the second vector. By planning the path and collecting key performance indicators, this application can have a more comprehensive understanding of the advantages and disadvantages of different inspection paths, so as to quickly determine the optimal inspection path. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 is a flow chart of Example 1 of the present application; Figure 2 This is a flowchart from the first segmentation in S51 to the first calculation in S53 in Example 2 of the present application; Figure 3 This is a flowchart of the second calculation from S54 retraining to S56 in Example 2 of the present application; Figure 4 It is a flowchart from the second calculation of S56 to the first modeling of S2 in Example 2 of the present application; Figure 5 This is a flow chart from S3 model training to S4 detection in Example 2 of the present application; Figure 6 is a flow chart of Example 3 of the present application; Figure 7 is a flow chart of Example 4 of the present application; Figure 8 This is a flow chart of Example 5 of the present application. DETAILED DESCRIPTION

[0029] The following combination Figures 1 to 8 This application is described in further detail.

[0030] Embodiment 1: This embodiment discloses a three-dimensional visual power inspection method, referring to Figure 1 The method includes: S1 data collection, S2 first modeling, S3 model training and S4 detection. The present application first collects real-time operation data and historical operation data, then establishes an inspection model, then uses historical operation data to train the inspection model, then inputs the real-time operation data into the trained inspection model, and outputs the first inspection result. The present embodiment includes the following steps: S1 data collection, including S11 first collection and S12 second collection.

[0031] S11 first acquisition, through specific sensors or data interfaces, can capture the operating parameters of the inspection target in real time, such as temperature, pressure, current, voltage, etc.

[0032] S12 The second collection, by querying the database or accessing the historical records, can collect the operation data of the inspection target in the past period of time, that is, the historical operation data.

[0033] S2 first modeling, based on the deep learning algorithm, establishes an inspection model, and the inspection model includes: a CNN sub-model, a BI-LSTM sub-model and a decision tree sub-model.

[0034] Through multi-layer convolution and pooling operations, convolutional neural networks (CNNs) can automatically learn from historical operation data and extract high-level features (i.e., operation features) that are useful for subsequent analysis.

[0035] Bidirectional long short-term memory network (BI-LSTM) can process sequence data and capture long-term dependencies in the data. In the inspection model, BI-LSTM is used to process the feature sequence extracted from CNN and further extract information from the feature sequence.

[0036] Decision trees are a simple yet powerful classification and regression method that makes predictions by learning decision rules from data. In the inspection model, decision trees are used to combine the outputs of CNN and BI-LSTM to ultimately output inspection results, such as device status, fault type, etc.

[0037] S3 model training includes S31 feature extraction, S32 first training and S33 second training to obtain a trained inspection model.

[0038] S31 feature extraction, uses the CNN sub-model to extract n running features.

[0039] S32 is the first training, inputting n running features into the BI-LSTM sub-model to obtain the output data of the BI-LSTM sub-model.

[0040] The BI-LSTM sub-model further extracts information that has an important impact on the inspection results by learning the time dependencies in the feature sequence.

[0041] S33 The second training inputs n running features and the output data of the BI-LSTM sub-model into the decision tree sub-model. The decision tree learns the relationship between these features and the target variable and finally outputs the decision result.

[0042] S4 detection, inputting the real-time operation data into the trained inspection model, and outputting the first inspection result.

[0043] This application uses a deep learning algorithm to establish an inspection model, which conducts in-depth analysis and processing of historical operation data through the inspection model to obtain the ability to judge real-time operation data, thereby achieving real-time monitoring and accurate detection of inspection targets.

[0044] Example 2: Reference Figure 2 The difference between this embodiment and the embodiment 1 is that the data acquisition in S1 further includes the third acquisition in S13: acquiring real-time images and historical images of the inspection target, wherein the historical images include normal images, abnormal images, and defect labels corresponding to the abnormal images; and after executing the data acquisition in S1 and before executing the first modeling in S2, it also includes: S51 is the first segmentation, which uses an image segmentation algorithm (such as a segmentation algorithm based on deep learning) to segment the real-time image to obtain multiple first segmentation blocks. By segmenting the image, the inspection target can be identified more accurately.

[0045] S52 is a first judgment, based on the first segmentation block, judging whether there is any occlusion in the inspection target. If so, executing S53 a first calculation; if not, executing S2 a first modeling.

[0046] The judgment based on the first segmentation block whether there is an obstruction in the inspection target can be made in the following manner: Scale information comparison: Based on the scale information of the inspection target, compare the actual size of the inspection target with the size in the image. If the actual size is significantly different from the size in the image, it means that the inspection target is blocked.

[0047] Edge information analysis: By detecting the edge contours of the inspection target, the continuity and closure between contours are analyzed. If there are discontinuous or incomplete edges, it means that the inspection target is blocked.

[0048] Color information analysis: By analyzing the color distribution of the inspection target, the connectivity and integrity of the color area are detected. If the color area is incomplete, it means that the inspection target is blocked.

[0049] S53 is a first calculation, in which the similarity between the real-time operation data and the historical operation data is calculated using a cosine similarity algorithm, and the historical operation data having the maximum similarity to the real-time operation data is used as the first data.

[0050] In the step of S4 detection, the first data is used to replace the real-time operation data and input into the inspection model.

[0051] For example, using the above solution to inspect equipment on a transmission line, the process is as follows: Real-time image acquisition: The camera carried by the drone can capture the running image of the inspection target in real time.

[0052] Historical image acquisition: The historical image data of the inspection target is obtained from the database, including images of normal operation, images of previous failures, and defect labels corresponding to these failure images (such as oil leakage, deformation, dirt, etc.).

[0053] Then the real-time image is segmented to obtain a plurality of first segmented blocks.

[0054] The edge detection algorithm is used to detect the edge contour of the object. Analysis shows that the edge contour is not continuous, indicating that the inspection target is blocked.

[0055] Then, the cosine similarity algorithm is used to calculate the similarity between the real-time operation data and the historical operation data, and the historical operation data when the similarity with the real-time operation data is the largest is used as the first data, and the first data is used to replace the real-time operation data and input into the inspection model.

[0056] This embodiment collects image data, and uses both image data and operation data to detect whether there is any obstruction to the inspection target. If the real-time image is obstructed, there is a risk that the real-time operation data may be incomplete. In order to reduce the risk of incomplete real-time operation data, historical operation data that is most similar to the real-time operation data is used instead of the real-time operation data for detection. That is, when the real-time operation data is missing a certain part of the data, the most similar and complete historical operation data is obtained by calculating the similarity with the historical operation data (in this embodiment, it is assumed that the historical operation data is complete data), and then the historical operation data is used to replace it in the inspection model, thereby improving the accuracy of the detection results.

[0057] In other embodiments, reference Figure 3 , after executing S53 the first calculation and before executing S2 the first modeling, it also includes: S54 retraining, using historical images to train the inspection model again to obtain a retrained inspection model. This embodiment can use historical images to enhance the image recognition capability of the inspection model, so that it can more accurately identify appearance changes and potential faults of the inspection target.

[0058] S55 is a first acquisition, in which all historical images corresponding to the first data are acquired through database query or file retrieval, and one historical image is recorded as a first image.

[0059] S56 is a second calculation, in which a hash algorithm (such as a mean hash algorithm aHash or a difference hash algorithm dHash) is used to respectively calculate the similarity between the real-time image and each first image, and by comparing the hash values ​​of the real-time image and each first image, the first image with the highest similarity is found and used as the second image.

[0060] Example of mean hash algorithm (aHash): Reduce size: Reduce the live image and the first image to a preset size, such as 8x8.

[0061] Convert to grayscale: Convert the reduced image to a 64-level grayscale image.

[0062] Calculate mean: Calculate the grayscale average of all pixels.

[0063] Binarization: Compare the grayscale value of each pixel with the grayscale average value. Set the grayscale value greater than or equal to the grayscale average value to 1, and set the grayscale value less than the grayscale average value to 0, and obtain a 64-bit hash value composed of 0 and 1.

[0064] Compare hash values: For two images, calculate their hash values ​​respectively, and then judge the similarity of the images by comparing the number of different bits in the two hash values. The fewer the number of different bits, the more similar the two images are.

[0065] Example of differential hash algorithm (dHash): Reduction in size and conversion to grayscale: Reduce the size of both images to 8x8 and convert them to 64-level grayscale.

[0066] Calculate the difference: For each row of adjacent pixels, calculate their grayscale value difference. For an 8x8 image, you will get 7 difference arrays (because each row has 8 pixels, but you can only calculate the difference of 7 adjacent pixels).

[0067] Binarization: For each difference array, if the difference is greater than or equal to a certain threshold (such as the mean or median), it is set to 1, otherwise it is set to 0, and 7 hash values ​​consisting of 0 and 1 are obtained.

[0068] Combined hash value: Combine these 7 hash values ​​into one long hash value, and concatenate them into a binary string to simplify comparison.

[0069] Compare hash values: Similar to the mean hashing algorithm, the similarity between the two images is determined by comparing the number of different bits in their hash values. The fewer the number of different bits in the hash values, the more similar the two images are.

[0070] In S4 detection, the second image and real-time running data are input into the retrained inspection model.

[0071] After obtaining the historical operation data that is most similar to the real-time operation data, this embodiment obtains the historical image corresponding to the historical operation data, calculates the similarity between the historical image and the real-time image, records the historical image that is most similar to the real-time image as the second image, and uses the second image to perform the detection step, thereby reducing the problem of the real-time image being blocked and unable to be inspected.

[0072] In other embodiments, reference Figure 4 , after executing the second calculation in S56 and before executing the first modeling in S2, it also includes: S57 second segmentation, using an image segmentation algorithm (such as a region-based segmentation algorithm, a feature-based segmentation algorithm, etc.) to segment the second image, separate different parts of the second image (such as the inspection target and the background), and extract a second segmentation block containing the inspection target.

[0073] S58 is the first fusion, in which the inspection target in the second segmentation block is used to replace the inspection target in the real-time image through a fusion algorithm, so as to generate a new real-time image including the complete inspection target and obtain a fused image.

[0074] In S4 detection, the fused images and real-time operation data are input into the retrained inspection model.

[0075] In this embodiment, the inspection target is segmented out from the second image, and the inspection target is fused into the real-time image. The fused real-time image can be used for detection, thereby improving the accuracy and reliability of the inspection result.

[0076] In other embodiments, reference Figure 5 , after executing S3 model training and before executing S4 detection, it also includes: S61: a fifth process of modifying the pixel value of each pixel point in the segmented block that does not include the inspection target to 0, so as to visually eliminate the background.

[0077] S62 is a second fusion, in which the segmented blocks including the inspection target and the segmented blocks after the pixel values ​​are modified are spliced ​​or fused according to their positions in the original image to obtain a fused real-time image, and the fused real-time image is used as a new fused image.

[0078] Fusion methods include simple superposition, weighted average, maximum fusion, etc.

[0079] This embodiment segments the image and identifies the segments containing the inspection target, then sets the pixel values ​​of the segments that do not contain the target to 0 to remove background interference, and finally merges these processed segments with the segments containing the target to generate a concise and clear real-time image focused on the inspection target, so as to improve the inspection efficiency and accuracy.

[0080] Example 3: Reference Figure 6 The difference between this embodiment and embodiment 2 is that after the first acquisition in S55 and before the second calculation in S56, the following is further included: S71: first integration, integrating the first image and the historical operation data corresponding to the first image into a training set.

[0081] S72 is the second modeling, which is to establish a generative adversarial network model, and use the training set to train the generative adversarial network model to obtain the trained generative adversarial network model. The trained GAN model can generate predicted images similar to real images.

[0082] S73 is the first prediction, which inputs the real-time running data into the trained generative adversarial network model and outputs multiple predicted images.

[0083] S74 adds a mask block, randomly adds a mask block to each predicted image, and uses the predicted image after the mask block is added as a new predicted image.

[0084] S75: third calculation, respectively calculating the similarity between the real-time image and each new predicted image, taking the new predicted image corresponding to the maximum similarity as the third image, and executing S4: detection.

[0085] In S4 detection, the third image is used instead of the second image to input into the retrained inspection model, and the second inspection result is output.

[0086] S76 sets a label and uses a target detection algorithm to add a corresponding label to the new predicted image.

[0087] S77 is the second judgment, judging whether the label in the step of setting the label is the same as the second inspection result. If so, it means that the model's recognition of the power system status is accurate, and then S1 data collection is executed; if not, S73 is executed. The first prediction.

[0088] This embodiment improves the accuracy and efficiency of inspections by continuously collecting data, training models, performing predictions and detections, and adjusting and optimizing based on the results.

[0089] Example 4: Reference Figure 7 The difference between this embodiment and embodiment 2 is that the step of data collection in S1 also includes, S14 fourth collection: using three-dimensional scanning technology (such as laser radar, structured light, etc.) to obtain three-dimensional shape information of the inspection target (such as power equipment, lines, etc.), and generate point cloud data, and collect real-time environmental data of the inspection target at the same time; and after executing S4 detection, it also includes: S81 The third modeling uses the collected 3D point cloud data to create a 3D model of the inspection target through 3D modeling software (such as AutoCAD, Blender, etc.).

[0090] S82 is the first processing, which deploys the real-time operation data (such as current, voltage, power, etc.) and the real-time environmental data (such as temperature, humidity, etc.) into the three-dimensional model, that is, mapping the real-time operation data and the real-time environmental data to the corresponding parts or attributes of the three-dimensional model to obtain a fused three-dimensional model.

[0091] S83 The second extraction uses computer vision technology (such as feature detection algorithm) to identify key points or features in the image, and finds points in the three-dimensional model that correspond to the feature points in the real-time image through matching or correspondence.

[0092] S84 is a second processing, in which the real-time image is aligned with the fused three-dimensional model using computer vision technology (such as stereo matching, image registration, etc.) to obtain an aligned three-dimensional model.

[0093] S85 runs simulation and uses simulation software (such as MATLAB / Simulink, ETAP, etc.) to run simulation on the aligned 3D model. The simulation software simulates the state and behavior of the inspection target in actual operation based on factors such as the model's geometry, material properties, operating data, and environmental data, thereby obtaining simulation results.

[0094] For example, during the inspection of a transformer, the simulation software will simulate the state and behavior of the transformer in actual operation based on the model's geometry (such as the size and structure of the transformer), material properties (such as the resistivity of the conductor, the dielectric constant of the insulating material, etc.), operating data (such as the input current and voltage waveform, etc.) and environmental data (such as the temperature and humidity of the surrounding air, etc.).

[0095] S86 is the third judgment, comparing the key indicators in the simulation result with the corresponding data in the first inspection result, judging whether the simulation result is consistent with the first inspection result, if so, executing S87 to output; if not, executing S88 to feedback.

[0096] S87 outputs the simulation result and the first inspection result.

[0097] S88 feedback: Feedback the simulation results and the first inspection results to the operation and maintenance personnel.

[0098] This embodiment establishes a three-dimensional model of the inspection target, deploys the real-time operation data and real-time environmental data into the three-dimensional model, and uses simulation software to simulate the fused three-dimensional model to obtain simulation results. By comparing the theoretical simulation results with the actual inspection results, it is determined whether the actual inspection results are accurate.

[0099] Example 5: Reference Figure 8 The difference between this embodiment and the fourth embodiment is that after executing the third judgment in S86 and before executing the output in S87, the following is further included: S91 plans the path, and sets m possible inspection paths according to the location of the inspection target (such as power equipment, lines, etc.) in the substation or power grid using the path planning algorithm.

[0100] S92 is a fifth collection process, in which the length, time and energy consumption of each inspection path are collected, and the length, time and energy consumption of each inspection path are integrated into a vector, which is recorded as a first vector.

[0101] Length refers to the distance the drone needs to travel from the starting point to the end point; time refers to the total time required to complete the inspection of the entire path; energy consumption refers to the energy consumed by the drone during the inspection process, such as electricity, fuel, etc.

[0102] S93 is a third processing, concatenating the m first vectors into a first matrix, where each row or column of the first matrix represents the length, time and energy consumption of an inspection path.

[0103] In order to eliminate the dimensional differences between different parameters, it is necessary to perform zero mean processing on the rows or columns of the first matrix, that is, normalize each parameter so that its mean is 0, and obtain the second matrix after zero mean processing.

[0104] S94 is a third calculation, calculating the eigenvalues ​​and eigenvectors of the second matrix, and recording the eigenvector corresponding to the maximum eigenvalue as the second vector.

[0105] The maximum eigenvalue means that the eigenvector occupies a dominant position in the matrix and can reflect the main characteristics of the matrix.

[0106] S95: fourth processing, traversing each element in the second vector, and obtaining the element with the largest value, which is recorded as the second data.

[0107] S96 sets a path, the position of the second data in the second vector corresponds to the index of the optimal patrol path, and the patrol path corresponding to the second data is used as the optimal patrol path.

[0108] The S87 output also includes outputting an optimal inspection path.

[0109] This embodiment plans multiple possible inspection paths according to the inspection target location, and then collects the length, time and energy consumption data of each path. Through data preprocessing and feature extraction, the eigenvalue and eigenvector analysis of the matrix is ​​used to find the inspection path with the best overall performance, that is, the path corresponding to the maximum eigenvalue, as the optimal inspection path for final execution, and output it to relevant personnel or systems to ensure the efficiency and accuracy of power system inspection.

[0110] The above are all preferred embodiments of the present application, and the protection scope of the present application is not limited thereto. Therefore, any equivalent changes made according to the structure, shape, and principle of the present application should be included in the protection scope of the present application.

Claims

1. A three-dimensional visual power inspection method, characterized in that: include: Data collection: including the first collection and the second collection; First collection: collect real-time operation data of inspection targets; Second collection: collect historical operation data of inspection targets; First modeling: Based on the deep learning algorithm, an inspection model is established, and the inspection model includes: a CNN sub-model, a BI-LSTM sub-model and a decision tree sub-model; Model training: including feature extraction, first training and second training to obtain the trained inspection model; Feature extraction: Use CNN sub-model to extract n running features; First training: input n running features into the BI-LSTM sub-model to obtain the output data of the BI-LSTM sub-model; Second training: input n running features and the output data of the BI-LSTM sub-model into the decision tree sub-model and output the decision results; Detection: Input the real-time operation data into the trained inspection model and output the first inspection result.

2. The three-dimensional visual power inspection method according to claim 1, characterized in that: The step of data collection also includes: a third collection: collecting real-time images and historical images of the inspection target, wherein the historical images include normal images, abnormal images, and defect labels corresponding to the abnormal images; and after executing the step of data collection and before executing the step of first modeling, it also includes: First segmentation: using an image segmentation algorithm to segment the real-time image to obtain a plurality of first segmentation blocks; First judgment: judging whether there is occlusion in the inspection target based on the first segmentation block, if so, executing the first calculation step; if not, executing the first modeling step; First calculation: using the cosine similarity algorithm to calculate the similarity between the real-time operation data and the historical operation data, and taking the historical operation data when the similarity with the real-time operation data is the maximum as the first data; In the detection step, the first data is used to replace the real-time operation data and input into the inspection model.

3. The three-dimensional visual power inspection method according to claim 2 is characterized in that: After the step of performing the first calculation and before the step of performing the first modeling, the method further includes: Retraining: Use historical images to train the inspection model again to obtain a retrained inspection model; First acquisition: acquiring all historical images corresponding to the first data, and recording one historical image as a first image; Second calculation: using a hash algorithm to calculate the similarity between the real-time image and each first image, and selecting the first image when the similarity with the real-time image is the maximum as the second image; In the detection step, the second image and the real-time operation data are input into the retrained inspection model.

4. The three-dimensional visual power inspection method according to claim 3 is characterized in that: After performing the second calculation step and before performing the first modeling step, the method further includes: Second segmentation: segmenting the second image using an image segmentation algorithm to obtain a second segmentation block containing the inspection target; First fusion: performing image fusion processing on the second segmented block containing the inspection target and the real-time image to obtain a fused image; In the detection step, the fused image and real-time operation data are input into the retrained inspection model.

5. The three-dimensional visualized power inspection method according to claim 3, characterized in that: After the model training step and before the detection step, the following steps are also included: The fifth process: modifying the pixel value of each pixel point in the segmented block that does not include the inspection target to 0; Second fusion: fusing the segmented blocks including the inspection target and the segmented blocks after the pixel values ​​are modified to obtain a fused real-time image, and using the fused real-time image as a new fused image.

6. The three-dimensional visualized power inspection method according to claim 3 or 4, characterized in that: After the first acquisition step and before the second calculation step, the method further includes: First integration: integrating the first image and the historical operation data corresponding to the first image into a training set; Second modeling: establish a generative adversarial network model, use the training set to train the generative adversarial network model, and obtain the trained generative adversarial network model; First prediction: input the real-time running data into the trained generative adversarial network model and output multiple predicted images; Add occlusion blocks: randomly add occlusion blocks to each predicted image, and use the predicted image after adding the occlusion blocks as the new predicted image; Third calculation: respectively calculating the similarity between the real-time image and each new predicted image, taking the new predicted image corresponding to the maximum similarity with the real-time image as the third image, and performing the detection step; In the detection step, the third image is used to replace the second image and input into the retrained inspection model to output the second inspection result.

7. The three-dimensional visualized power inspection method according to claim 6, characterized in that: After the detection step is performed, it also includes: Set labels: Use target detection algorithms to add corresponding labels to new predicted images; Second judgment: judge whether the label in the step of setting the label is the same as the second inspection result. If so, execute the step of data collection; if not, execute the step of first prediction.

8. The three-dimensional visualized power inspection method according to any one of claims 2 to 4, characterized in that: The data collection step also includes a fourth collection: collecting three-dimensional point cloud data and real-time environmental data of the inspection target; And after the step of performing the detection, it also includes: Third modeling: Use 3D point cloud data to create a 3D model of the inspection target; First processing: deploying the real-time operation data and the real-time environment data into the three-dimensional model to obtain a fused three-dimensional model; Run simulation: Use simulation software to run simulation on the fused 3D model to obtain simulation results; Third judgment: judging whether the simulation result is consistent with the first inspection result, if so, executing the output step; if not, executing the feedback step; Output: Output simulation results and the first inspection results; Feedback: Feedback the simulation results and the first inspection results to the operation and maintenance personnel.

9. The three-dimensional visualized power inspection method according to claim 8, characterized in that: After executing the step of the first processing and before executing the step of running the simulation, the method further includes: Second extraction: extracting feature points of the inspection target in the real-time image and marking corresponding points of the feature points in the three-dimensional model; Second processing: using computer vision technology to align the real-time image with the fused 3D model to obtain an aligned 3D model; In the step of running the simulation, the aligned three-dimensional model is used instead of the fused three-dimensional model to run the simulation.

10. The three-dimensional visualized power inspection method according to claim 9, characterized in that: After executing the third judgment step and before executing the output step, the method further includes: Planning paths: According to the location of the inspection target, set m inspection paths; Fifth collection: collect the length, time and energy consumption of each inspection path, and integrate the length, time and energy consumption of each inspection path into a vector, which is recorded as the first vector; The third process: concatenate the m first vectors into a first matrix, perform zero mean processing on the rows or columns of the first matrix, and obtain a second matrix; The third calculation: calculate the eigenvalue and eigenvector of the second matrix, and record the eigenvector corresponding to the maximum eigenvalue as the second vector; Fourth process: traverse each element in the second vector, and obtain the element with the largest value, which is recorded as the second data; Set path: take the inspection path corresponding to the second data as the optimal inspection path; The outputting step also includes outputting an optimal inspection path.

Images