A new intelligent inspection system for power equipment based on a large model

Abstract

The application discloses a new type of power equipment intelligent inspection system based on a large model, which comprises a multi-modal data module, a degree detection module, a quantitative analysis module and a cross-modal enhancement module; the multi-modal data module is used for non-contact acquisition of multi-dimensional data, and trigger fusion and time synchronization of the multi-dimensional data; the degree detection module can utilize the prior knowledge of the pre-training matching large model, obtain limited sample data guided visual feature matching through the generation of descriptive text, identify multi-dimensional data outliers in the feature space as potential unknown aging candidates by using the bipartite graph matching technology, and output the insulation aging degree; the quantitative analysis module calculates the one-way distance value between the insulation aging degree point cloud and the target point cloud. The application can actively discover and mark the aging degree data that does not appear in the training set, fundamentally expands the aging degree detection range of the system, and improves the identification ability of the equipment abnormity in the booster station.

Description

A novel intelligent inspection system for power equipment based on a large model Technical Field

[0001] This invention relates to the field of power equipment inspection technology, and specifically to a novel intelligent power equipment inspection system based on a large model. Background Technology

[0002] With the continuous expansion of power systems, power substations, as key power distribution facilities, directly impact the reliability and security of the power grid. To ensure the stable operation of power substations, regular inspections of various power equipment have become crucial. In recent years, intelligent inspection systems have been gradually applied to the automated inspection of power substations. By collecting multimodal data such as visible light, infrared, partial discharge, and acoustic signatures, these systems can initially identify functional aging issues within the substations, such as abnormal appearances, localized overheating, and mechanical loosening.

[0003] However, most current intelligent inspection systems for power equipment typically require a large amount of labeled data for training. However, the degree of insulation aging anomalies in substation equipment varies in actual operation, resulting in a scarcity of positive samples and making it difficult for the system to effectively identify these anomalies during inspections. Summary of the Invention

[0004] Therefore, this invention provides a novel intelligent inspection system for power equipment based on a large model to solve the problems in the prior art.

[0005] To achieve the above objectives, the present invention provides the following technical solution:

[0006] A novel intelligent inspection system for power equipment based on a large model includes a multimodal data module, a degree detection module, a quantitative analysis module, and a cross-modal enhancement module.

[0007] The multimodal data module is used for non-contact acquisition of multidimensional data and for trigger-based fusion and time synchronization of the multidimensional data;

[0008] The degree detection module can utilize the prior knowledge of the pre-trained matching large model, obtain limited sample data by generating descriptive text to guide visual feature matching, and use bipartite graph matching technology to identify multidimensional data outliers in the feature space as potential unknown aging candidates, and output the degree of insulation aging.

[0009] The quantitative analysis module calculates the one-way distance between the insulation aging degree point cloud and the target point cloud, and takes the maximum value of the distance; then it combines the maximum value with the time dimension to form a state evolution sequence.

[0010] The cross-modal enhancement module is used to input limited sample data and the maximum value of one-way distance as constraints into a pre-trained generative large model. The generative large model can generate diverse aging synthetic data to form an enhanced dataset.

[0011] Furthermore, the multidimensional data is constructed based on sensor subunits, which include 2D high-definition visible light cameras and 3D lidar.

[0012] The 2D high-definition visible light camera is used to capture surface information of the device.

[0013] The 3D lidar is used to generate point cloud data.

[0014] Furthermore, the triggered fusion is built on an event-driven collaborative acquisition mechanism. When a sensor identifies a specific target, it immediately generates a trigger signal, which is then synchronously woken up and guided by the central controller to other heterogeneous sensors.

[0015] Furthermore, the degree detection module includes a large model preset subunit, a learning subunit, and a region subunit;

[0016] The large model preset subunit can construct and train a matching large model based on text-image pairs of power equipment inspection data on the Internet, combined with a neural network architecture.

[0017] The learning subunit is able to use a pre-trained large model to generate descriptive text related to the image content as limited sample data when faced with a small number of device images to be detected, so as to summarize the key features and potential aging information in the image.

[0018] The aforementioned molecular unit can map the aging degree features of the device image to a high-dimensional feature space. Then, based on bipartite graph matching technology, the feature points in the device image feature space are input with the known aging degree features, and the similarity between the feature points and the known aging degree features is calculated. If a certain feature point has a similarity to all known aging degree features If all values ​​are below the set threshold, they are identified as outliers.

[0019] Similarity The calculation formula is as follows:

[0020]

[0021] in, This is a characteristic of the aging degree of the device image. Given the known aging characteristics, this represents the prototype representation of the known aging levels in the training set.

[0022] Furthermore, the specific content of the pre-defined sub-unit of the large model is as follows:

[0023] 1) Collect a large number of text-image pairs of power equipment inspection data from the Internet to construct a training set;

[0024] 2) Model architecture selection: A neural network structure is selected as the model architecture, and unsupervised learning is used to enable the model to learn to associate similar text and images to form a basic concept mapping;

[0025] 3) Fine-tuning and optimization process: Further adjust parameters for specific application scenarios to improve the model's ability to understand fine-grained differences and its generalization level;

[0026] 4) Evaluation and verification stage: Regularly test the performance of the model and make necessary corrections and improvements based on feedback information until a satisfactory result is achieved and a matching large model is obtained.

[0027] Furthermore, the region molecular unit can also calculate the similarity cost between each feature point and all nodes with known aging levels. A fixed matching cost threshold is set for virtual nodes; if the similarity cost of a feature point with all known aging levels is higher than the threshold, the feature point is preferentially matched to the virtual node, thereby avoiding forcibly classifying it into the known aging level and reducing misjudgment.

[0028] Similarity cost The calculation formula is as follows:

[0029]

[0030] Where N is the number of feature points, and M is the number of known aging levels. It is an assignment function, representing the allocation of feature points. Assigned to degree Or virtual nodes, It is the fixed matching cost threshold for virtual nodes. It is an indicator function.

[0031] Furthermore, the specific content of the quantitative analysis module is as follows:

[0032] 1) Retrieve the reference point cloud data of the 3D CAD model of the corresponding circuit breaker and disconnector from the standard library, and define it as the centerline;

[0033] 2) Compare the target point cloud of insulation aging degree data output by the degree detection module with the model, calculate the one-way distance from each point in the real-time point cloud to the nearest point in the target point cloud, and take the maximum value of the distance. maximum value The calculation formula is as follows:

[0034]

[0035] Where A is the point cloud set of aging degree data, B is the target point cloud set, a is any point cloud in the aging degree data, and b is the target point cloud. Let be the Euclidean distance between points a and b;

[0036] 3) Combine the maximum value with the time dimension, record the maximum value calculated for the same real-time data in multiple consecutive inspection cycles, and form a state evolution sequence.

[0037] Furthermore, when the constraints are input into the generative large model, the model can generate large-scale variant aging synthetic data with the same aging degree under different environmental conditions. All generated aging synthetic data will together with a small number of real samples to form an augmented dataset. The augmented dataset can perform incremental training on the degree detection module periodically or triggered.

[0038] The present invention has the following advantages: By utilizing the prior knowledge of the pre-trained matching large model, the present invention can quickly establish accurate classification boundaries with a small number of samples through text-guided visual feature matching. It can actively discover and mark aging degree data that has not appeared in the training set, fundamentally expanding the aging degree detection range of the system and improving the ability to identify abnormalities in equipment within the booster station.

[0039] Meanwhile, by synthesizing diverse abnormal data through the cross-modal enhancement module, the lack of rare fault samples in power substations is compensated for; combined with the state evolution sequence generated by the quantitative analysis module, a quantitative assessment of the abnormal development trend of equipment is achieved, providing a reliable basis for insulation aging analysis.

[0040] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. Attached Figure Description

[0041] To more intuitively illustrate the prior art and this application, exemplary drawings are provided below. It should be understood that the specific shapes and structures shown in the drawings should not generally be regarded as limiting conditions for implementing this application; for example, based on the technical concept disclosed in this application and the exemplary drawings, those skilled in the art are able to easily make conventional adjustments or further optimizations to the addition / reduction / classification, specific shapes, positional relationships, connection methods, size ratios, etc. of certain units (components).

[0042] Figure 1 is a block diagram of a novel intelligent inspection system for power equipment based on a large model according to the present invention. Detailed Implementation

[0043] The following specific embodiments illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. It should be understood that these embodiments are merely for further explanation of the present invention and should not be construed as limiting the scope of protection of the present invention. Technical engineers in the field can make some non-essential improvements and adjustments to the present invention based on the above-described content. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0044] Please refer to Figure 1. A novel intelligent inspection system for power equipment based on a large model includes a multimodal data module, a degree detection module, a quantitative analysis module, and a cross-modal enhancement module.

[0045] The multimodal data module integrates various types of sensors for non-contact acquisition of multidimensional data, and performs triggered fusion and time synchronization of the multidimensional data, which facilitates the comprehensive and accurate conversion of the power equipment status into digital information that can be understood and processed by the computing system.

[0046] The various sensor subunits include 2D high-definition visible light cameras and 3D lidar. The 2D high-definition visible light cameras are used to capture the degree of contamination on the surface of transformer bushings and the oxidation and corrosion of circuit breaker contacts; the 3D lidar is used to generate point cloud data to provide information on the displacement deviation of the equipment's circuit breaker operating mechanism.

[0047] Triggered fusion is built on an event-driven collaborative acquisition mechanism: when a master sensor (such as a 2D high-definition camera) identifies a specific target or suspected anomaly (such as discharge marks on the surface of a transformer bushing), it immediately generates a trigger signal. This signal is synchronously woken up by the central controller and guides other heterogeneous sensors (such as a 3D LiDAR to perform a fine scan of the area and a voiceprint sensor to collect the audio spectrum at the corresponding moment), so as to achieve precise linkage of multiple sensors in time and space.

[0048] The multimodal data module also includes a high-precision clock source. Before detection, a unique timestamp is assigned to each sensor using a high-precision clock source (such as GPS timing or an atomic clock), and during the initialization phase, the local clocks of all devices are calibrated to be consistent with the global reference time. When an event triggers a 3D scan, the main control unit immediately sends a synchronization signal with a precise timestamp to all relevant sensors, instructing them to begin recording data. Upon receiving this signal, each sensor adjusts its actual sampling start point according to its internally preset time delay compensation mechanism to eliminate deviations caused by network transmission latency and hardware processing time, thereby forming a multi-dimensional dataset within a time slice for the same event.

[0049] The multimodal data module also integrates an infrared thermal imager for detecting equipment overheating anomalies.

[0050] The aging degree detection module leverages prior knowledge from a pre-trained matching model to guide visual feature matching using limited sample data obtained through descriptive text generation. It employs bipartite graph matching technology to identify multidimensional data outliers in the feature space as potential unknown aging candidates. Furthermore, it avoids misjudging known aging degrees by introducing virtual nodes to calculate matching costs, outputting the insulation aging degree, covering both known and potentially unknown aging degrees. It identifies the aging degree of known equipment types and proactively discovers novel or rare anomalies not seen during training, significantly expanding the system's aging degree detection coverage and robustness in practical applications.

[0051] The severity detection module includes a large model preset subunit, a learning subunit, and a region subunit. The large model preset subunit can construct and train a matching large model based on text-image pairs of power equipment inspection data from the internet, combined with a neural network architecture. Details are as follows:

[0052] 1) First, collect a large number of text-image pairs of power equipment inspections from the Internet to build a training set.

[0053] 2) Model architecture selection: A neural network structure is selected as the model architecture, and unsupervised learning is used to teach the model to associate similar texts and images to form a basic concept mapping.

[0054] 3) Fine-tuning and optimization process: Further adjust parameters for specific application scenarios to improve the model's ability to understand fine-grained differences and its generalization level.

[0055] 4) Evaluation and verification stage: Regularly test the performance of the model and make necessary corrections and improvements based on feedback until satisfactory results are achieved.

[0056] The aforementioned neural network architectures broadly refer to various deep learning architectures used to process and understand multimodal data, such as Transformers, which effectively capture long-range dependencies in sequential data through self-attention mechanisms; and Convolutional Neural Networks (CNNs), which are particularly suitable for image recognition tasks and can automatically extract spatial hierarchical features. These models are widely used to learn complex representations from various types of information, such as text and vision, thereby supporting a range of advanced cognitive functions such as classification, detection, and generation.

[0057] When faced with a small number of device images to be detected, the learning subunit can use a pre-trained large model to generate descriptive text related to the image content as limited sample data to summarize the key features and potential aging information in the image.

[0058] The matching model matches images with the semantic features contained in descriptive text. Through this fusion of text and visual features, it can accurately output the aging features in the image, thereby achieving high recognition accuracy under limited data conditions.

[0059] Descriptive text such as "a picture showing cracks on the surface of a transformer bushing" or "a picture of a circuit breaker in its normal closed state".

[0060] The molecular unit can map the aging characteristics of a device image into a high-dimensional feature space. In this feature space, each feature point represents a local feature in the image. Simultaneously, known aging characteristics are also mapped into the same feature space, forming a set of known aging characteristics.

[0061] Then, based on bipartite graph matching technology, feature points in the device image feature space and known aging degree features are input, where nodes on one side represent feature points in the device image feature space, and nodes on the other side represent known aging degree features. The similarity between feature points and known aging degree features is then calculated. The process involves determining the edges and their similarities in a bipartite graph. Higher similarity results in a larger edge weight, indicating a greater likelihood that the feature point belongs to a known aging level. If the similarity between a feature point and all known aging level features is below a set threshold (i.e., no effective match can be formed), it is identified as an outlier. Outliers are marked as potential unknown aging candidates due to the lack of a known aging level correspondence, and the insulation aging level is output. (Similarity) The calculation formula is as follows:

[0062]

[0063] in, This is a characteristic of the aging degree of the device image. Given the known aging characteristics, this represents the prototype representation of the known aging levels in the training set.

[0064] Simultaneously, virtual nodes are introduced, and the similarity cost between each feature point and all nodes with known aging levels is calculated. A fixed matching cost threshold is set for virtual nodes. If the similarity cost of a feature point with all known aging levels is higher than this threshold (i.e., the similarity is low), the feature point is preferentially matched to the virtual node, thus avoiding forced classification into known aging levels and effectively reducing misclassification. Similarity cost The calculation formula is as follows:

[0065]

[0066] Where N is the number of feature points, and M is the number of known aging levels. It is an assignment function, representing the allocation of feature points. Assigned to degree Or a virtual node (unknown). It is the fixed matching cost threshold for virtual nodes. It is an indicator function; its value is 1 when the condition is true, and 0 otherwise.

[0067] The quantitative analysis module calculates the unidirectional distance between the insulation aging degree point cloud and the target point cloud, and takes the maximum value of the distance. This maximum value is then combined with the time dimension to form a state evolution sequence, facilitating tracking and analysis. By observing its evolution trend, the speed of aging progression can be assessed. This transforms qualitative aging degree identification into quantitative state assessment and dynamic tracking, providing accurate data for predictive maintenance decisions. Specific details are as follows:

[0068] 1) Retrieve the reference point cloud data of the 3D CAD model of the corresponding circuit breaker and disconnector from the standard library, and define it as the centerline.

[0069] 2) Compare the target point cloud of insulation aging degree data output by the degree detection module with the model, calculate the one-way distance from each point in the real-time point cloud to the nearest point in the target point cloud, and take the maximum value of the distance. This quantifies the maximum local deviation. (Maximum value) The calculation formula is as follows:

[0070]

[0071] Where A is the point cloud set of aging degree data, B is the target point cloud set, a is any point cloud in the aging degree data, and b is the target point cloud. Let be the Euclidean distance between points a and b.

[0072] Euclidean distance The calculation formula is as follows:

[0073]

[0074] in, Let a be the three-dimensional coordinates of point a. Let b be the three-dimensional coordinates of point b.

[0075] 3) By combining the maximum value with the time dimension, the maximum value calculated for the same real-time level data over multiple consecutive inspection cycles is recorded to form a state evolution sequence. By analyzing the trend of this sequence (such as stability, linear growth, or accelerated expansion), the evolution speed and severity level of the aging level can be determined.

[0076] The cross-modal augmentation module is used to input limited sample data and the maximum value of one-way distance as constraints into a pre-trained generative large model. The generative large model can generate diverse synthetic aging data, forming an ever-enriched augmentation dataset. The synthetic aging data is used to compensate for the scarce aging samples in the real world and integrates multimodal information to enhance the generalization ability and the level of awareness of rare cases, forming a closed loop of self-evolution for the system.

[0077] During pre-training, generative large models are first trained on massive unlabeled multimodal datasets using a self-supervised learning objective, allowing the model to learn the underlying patterns in the data. This process uses mask reconstruction as the objective, enabling the model to grasp the distribution of relationships between visual features, physical structures, and semantic concepts, thereby acquiring general content generation capabilities and prior knowledge.

[0078] When constraints are input into a generative large-scale model, the model can generate large-scale variant aging synthetic data with the same aging degree under different environmental conditions. Ultimately, all generated aging synthetic data, together with a small number of real samples, constitute a continuously expanding augmented dataset. This dataset is systematically used to incrementally train the degree detection module periodically or triggered by specific events. In this way, the system forms a self-evolving closed loop from "discovering new knowledge" to "creating knowledge" and then feeding back into "cognitive ability," enabling the entire system to become increasingly intelligent over time, and its identification and assessment of rare aging degrees becoming increasingly accurate.

[0079] For example, it can generate high-fidelity infrared thermograms of the circuit breaker contacts erosion under different load currents and ambient temperatures, and generate point cloud data of the entire evolution sequence of the crack from initial initiation to severe propagation.

[0080] The aforementioned intelligent inspection system for power equipment based on a large model utilizes the prior knowledge of a pre-trained matching large model and establishes accurate classification boundaries quickly with a small number of samples through text-guided visual feature matching. It can proactively discover and label aging data that has not appeared in the training set, fundamentally expanding the aging detection range of the system and improving the ability to identify equipment anomalies in the substation.

[0081] Meanwhile, by synthesizing diverse abnormal data through the cross-modal enhancement module, the lack of rare fault samples in power substations is compensated for; combined with the state evolution sequence generated by the quantitative analysis module, a quantitative assessment of the abnormal development trend of equipment is achieved, providing a reliable basis for insulation aging analysis.

[0082] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A novel intelligent inspection system for power equipment based on a large-scale model, characterized in that, It includes a multimodal data module, a degree detection module, a quantization analysis module, and a cross-modal enhancement module. The multimodal data module is used for non-contact acquisition of multidimensional data and performs triggered fusion and time synchronization of the multidimensional data. The degree detection module can utilize the prior knowledge of a pre-trained matching large model to guide visual feature matching by generating descriptive text to obtain limited sample data. It uses bipartite graph matching technology to identify outliers in the feature space as potential unknown aging candidates and outputs the degree of insulation aging. The degree detection module includes a large model preset subunit, a learning subunit, and a region subunit. The unit can construct and train a matching large model based on text-image pairs of power equipment inspection data from the Internet, combined with a neural network architecture. The learning subunit, when faced with a small number of equipment images to be detected, can use the pre-trained large model to generate descriptive text related to the image content as limited sample data, summarizing key features and potential aging information in the image. The partitioning subunit can map the aging features of the equipment image to a high-dimensional feature space, and then, based on bipartite graph matching technology, input the feature points in the equipment image feature space with known aging features, calculating the similarity between the feature points and the known aging features. If a certain feature point has a similarity to all known aging degree features If all similarities are below a set threshold, they are identified as outliers; The calculation formula is as follows: in, This is a characteristic of the aging degree of the device image. Given the known aging degree characteristics, the prototype representation of the known aging degree in the training set is used. The quantification analysis module calculates the one-way distance between the insulation aging degree point cloud and the target point cloud, and takes the maximum value of the distance. The maximum value is combined with the time dimension to form a state evolution sequence. The specific contents of the quantification analysis module are as follows: 1) Retrieves the reference point cloud data under the 3D CAD model of the corresponding circuit breaker and disconnector from the standard library, and defines it as the centerline; 2) Compares the target point cloud of the insulation aging degree data output by the degree detection module with the model, calculates the one-way distance from each point in the real-time point cloud to the nearest point in the target point cloud, and takes the maximum value of the distance. maximum value The calculation formula is as follows: Where A is the point cloud set of aging degree data, B is the target point cloud set, a is any point cloud in the aging degree data, and b is the target point cloud. 3) Combine the maximum value with the time dimension, record the maximum value calculated for the same real-time data in multiple consecutive inspection cycles, and form a state evolution sequence. The cross-modal enhancement module is used to input the finite sample data and the maximum value of the one-way distance as constraints into the pre-trained generative large model. The generative large model can generate diverse aging synthetic data to form an enhanced dataset.

2. The novel intelligent inspection system for power equipment based on a large model according to claim 1, characterized in that, The multidimensional data is constructed based on sensor subunits, which include a 2D high-definition visible light camera and a 3D lidar; the 2D high-definition visible light camera is used to capture the appearance information of the device surface; the 3D lidar is used to generate point cloud data.

3. The novel intelligent inspection system for power equipment based on a large model according to claim 1, characterized in that, The trigger-based fusion is built on an event-driven collaborative acquisition mechanism. When a sensor detects a specific target, it immediately generates a trigger signal, which is then synchronously woken up and guided by the central controller to other heterogeneous sensors.

4. The novel intelligent inspection system for power equipment based on a large model according to claim 1, characterized in that, The specific content of the pre-set sub-units of the large model is as follows: 1) Collect a large number of text-image pairs of power equipment inspection from the Internet to construct a training set; 2) Model architecture selection: Select a neural network structure as the model architecture, and then use unsupervised learning to let the model learn to associate similar text and images to form a basic concept mapping; 3) Fine-tuning and optimization process: Further adjust the parameters for specific application scenarios to improve the model's ability to understand fine-grained differences and its generalization level; 4) Evaluation and verification stage: Regularly test the performance of the model, and make necessary corrections and improvements based on feedback information until a satisfactory effect is achieved, thus obtaining a matching large model.

5. The novel intelligent inspection system for power equipment based on a large model according to claim 1, characterized in that, The region molecular unit can also calculate the similarity cost between each feature point and all nodes with known aging levels. A fixed matching cost threshold is set for virtual nodes. If the similarity cost of a feature point with all known aging levels is higher than this threshold, the feature point is preferentially matched to a virtual node, thus avoiding forced classification into known aging levels and reducing misclassification. The calculation formula is as follows: Where N is the number of feature points, and M is the number of known aging levels. It is an assignment function, representing the allocation of feature points. Assigned to degree Or virtual nodes, It is the fixed matching cost threshold for virtual nodes. It is an indicator function.

6. The novel intelligent inspection system for power equipment based on a large model according to claim 1, characterized in that, When the constraints are input into the generative large model, the model can generate large-scale variant aging synthetic data with the same aging degree under different environmental conditions. All generated aging synthetic data will be combined with a small number of real samples to form an augmented dataset. The augmented dataset can be used to incrementally train the degree detection module periodically or triggered.

Images