Inspection sample image data enhancement method for electric power artificial intelligence platform

Abstract

The invention relates to the technical field of intelligent operation and maintenance of an electric power system, in particular to an inspection sample image data enhancement method for an electric power artificial intelligence platform, which is used for solving the problems that in the prior art, history and equipment knowledge cannot be fused to construct a forbidden area, a co-occurrence rule and component association, cross-component defect positions cannot be effectively adjusted, and the detection accuracy is poor. Defect distribution is difficult to accurately control, and physical rationality and engineering credibility are reduced. According to the method, a forbidden area, a co-occurrence rule and component association are constructed by fusing history and equipment knowledge, masks are generated through kernel density estimation to suppress invalid defects, co-occurrence frequencies are counted based on feature vectors, co-occurrence relationships are determined by combining distances and similarities, and the masks, matrixes and graphs are embedded and coded into conditional vectors, so that the non-ineffective defects are suppressed. And zero setting is performed on a forbidden area in the generative network, illegal co-occurrence is filtered, and cross-component defect positions are adjusted, so that defect distribution is accurately controlled, and physical rationality and engineering credibility are enhanced.

Description

Technical Field

[0001] This invention relates to the field of intelligent operation and maintenance technology for power systems, and more specifically, to a method for enhancing inspection sample image data for power artificial intelligence platforms. Background Technology

[0002] In intelligent power system inspection, deep learning models are prone to overfitting and missed detections due to the scarcity of fault samples and the variability of the environment. General data augmentation methods have limited sample quality due to a lack of understanding of power scenarios, while advanced generation techniques face problems such as unstable training and high computational cost, making it difficult to meet the needs of practical applications.

[0003] Reference patent application CN111968048A discloses a method and system for enhancing few-sample image data in power line inspections, including: calibrating the equipment area in a small number of images of the object to be identified collected during power line inspections; removing the background from the calibrated images and transforming the background-removed object images to simulate the situation of taking pictures of the equipment from different angles and distances; updating the background image to obtain images of the object to be identified in different backgrounds, thereby generating a large number of calibrated images; achieving few-sample enhancement, which is convenient for training using artificial intelligence deep learning algorithms and is beneficial for accurate image recognition;

[0004] However, the aforementioned reference patents generate virtual background textures and perform scaling, rotation, and affine transformations to automatically expand the background and scale diversity of samples, effectively reducing the cost of manual annotation and image collection, and improving the accuracy of robot inspection image recognition under conditions of few samples. However, they cannot integrate historical and equipment knowledge to construct prohibited areas, co-occurrence rules, and component associations, cannot effectively adjust the location of defects across components, and have difficulty accurately controlling the distribution of defects, thus reducing physical rationality and engineering credibility. At the same time, they cannot integrate multi-scale features, cross-modal stitching, and environmental conditions, cannot achieve collaborative generation of visible light and infrared images, and cannot ensure authenticity through a discriminator combined with L1 and perceptual loss, thus reducing the generation quality and practicality.

[0005] To address these issues, we propose a method for enhancing inspection sample image data for power artificial intelligence platforms. Summary of the Invention

[0006] The purpose of this invention is to provide a method for enhancing inspection sample image data for power artificial intelligence platforms. This method solves the problems of existing technologies, which cannot integrate historical and equipment knowledge to construct restricted areas, co-occurrence rules, and component associations, cannot effectively adjust the location of defects across components, and are difficult to accurately control the distribution of defects, thus reducing physical rationality and engineering credibility. At the same time, these technologies cannot integrate multi-scale features, cross-modal stitching, and environmental conditions, cannot achieve collaborative generation of visible light and infrared images, and cannot ensure authenticity through a discriminator combined with L1 and perceptual loss, thus reducing the quality and practicality of the generated images.

[0007] The objective of this invention is achieved through the following technical solution:

[0008] A method for enhancing inspection sample image data for power artificial intelligence platforms includes the following steps:

[0009] Step 1: Collect inspection image data, environmental monitoring data, and spatial positioning data generated during high-altitude inspection, on-site close-range inspection, and continuous monitoring of fixed areas of power equipment. Preprocess the collected data to generate defect labeling data for typical defect areas in the images. Align the images, labels, and non-image data through timestamps and spatial locations to construct a structured initial sample set containing visual content, defect labels, and environmental and spatial conditions.

[0010] Step 2: Using equipment type, ambient temperature, ambient humidity, light intensity, three-dimensional geographic coordinates, and equipment attitude angle, combined with thermodynamic and optical response laws, train and generate a network to synthesize defective images;

[0011] Step 3: Based on the spatial distribution characteristics of defect annotations and the topological relationship of components, construct defect location forbidden areas, morphological co-occurrence rules and component association constraints, encode regularized condition vectors and embed them into the generative network to regulate the spatial location, morphology and cross-component association of defects;

[0012] Step 4: Extract multi-scale features from visible light images and infrared thermal images, fuse cross-modal representations, and combine device type, ambient temperature, ambient humidity, light intensity, and component-level semantic constraints to generate paired visible light and infrared defect images through a generative network.

[0013] In a preferred embodiment of the present invention, step two, which utilizes equipment type, ambient temperature, ambient humidity, light intensity, three-dimensional geographic coordinates, and equipment attitude angle, combined with thermodynamic and optical response laws, includes:

[0014] The device type, ambient temperature, ambient humidity, light intensity, 3D geographic coordinates, and device attitude angle are obtained as input parameters. The device type is extracted from the predefined category set and converted into a one-hot encoded vector with a length equal to the total number of categories. The one-hot encoded vector is mapped to a continuous real number vector of fixed dimension through an embedding layer. The ambient temperature and ambient humidity are standardized, and the light intensity is linearly normalized.

[0015] The device installation attitude angle is expressed in radians. The sine and cosine values ​​of the attitude angle are calculated to form a two-dimensional real vector. The embedded vector of the device type, the standardized ambient temperature, the standardized ambient humidity, the normalized light intensity, the sine value of the attitude angle, and the cosine value of the attitude angle are connected in sequence to generate a real vector of fixed length.

[0016] Calculate the heat generated by conductive components, calculate the heat transfer rate inside the device, calculate the radiant energy density on the surface of the device, calculate the pixel brightness value of the infrared image, and calculate the energy reflected by the device surface from incident light.

[0017] The device type embedding vector, environmental parameters, attitude angle representation, heat generation, heat transfer rate, radiative energy density, pixel brightness value, and reflected energy are all used as input parameters and input into the corresponding module of the generation model.

[0018] In a preferred embodiment of the present invention, the process of training the network to synthesize defective images in step two includes:

[0019] Acquire a real infrared image, device type, ambient temperature, ambient humidity, light intensity, device attitude angle, and defect type. Perform a T-step noise addition operation on the real infrared image. After T steps, the image becomes a pure noise image with a standard normal distribution.

[0020] The defect type is taken from the predefined category set and converted into a one-hot encoded vector. The vector length is equal to the total number of defect categories. The one-hot encoded vector of the defect type, the embedded vector of the equipment type, the normalized ambient temperature, the normalized ambient humidity, the normalized light intensity, the sine value of the attitude angle, and the cosine value of the attitude angle are connected in sequence to generate a real number vector of fixed length.

[0021] At each time step of the denoising process, the noisy image, time step number, and extended conditional input vector are input into the generator network. The generator network operates in an encoder-decoder structure. In each layer, the extended conditional input vector information is fused through a cross-attention mechanism, and the output is an estimate of the noise added at the current time step. Using the noise estimate, the noisy image, and fixed coefficients in the noise scheduling sequence, the denoised image is calculated according to the denoising update rule, and the denoised image is output.

[0022] The main loss term, physical consistency loss term, and perceptual loss term are calculated. The main loss term, physical consistency loss term, and perceptual loss term are then summed by weighting them with fixed weight coefficients, which are constants set before training, to obtain the total loss value. Based on the total loss value, the parameters of the generator network are updated using gradient descent.

[0023] In a preferred embodiment of the present invention, the process of constructing the defect location forbidden region and morphological co-occurrence rule in step three includes:

[0024] Read the spatial locations of all historical defect samples from the historical database, set the bandwidth parameter h for kernel density estimation, and set the parameter value as a constant manually set before training. For each calculation location x in three-dimensional space, use the formula to calculate the defect generation probability density value at location x.

[0025] Read the geometric coordinates from the equipment design model, extract the spatial range of the key engineering area, mark all spatial points within the key engineering area, perform corresponding operations to construct the restricted area, and set the defect generation probability density value of all spatial points within the restricted area to zero before generating the output results.

[0026] For each type of defect, a four-dimensional feature vector is constructed, and corresponding processing operations are performed based on historical data. For any two defects A and B, the co-occurrence determination result is calculated using a function, and the forbidden region definition and defect co-occurrence determination rules are output as constraints for the generative model.

[0027] In a preferred embodiment of the present invention, the process of constructing component association constraints in step three includes:

[0028] Read the geometric coordinates of seven structural units from the equipment design model, determine the fixed spatial area of ​​each component, construct an undirected connection graph between components based on the mechanical assembly relationship, add an edge to the graph if two components have a mechanical connection, and read the strength reference value of each connection type from the connection parameter table.

[0029] For each pair of connected components B i With B j Perform the corresponding processing operations, construct a static mapping table, with the propagation weight as the input and the distance threshold as the output. For each edge in the graph, if its propagation weight is greater than zero, then perform the corresponding processing operations.

[0030] For each pair of connected components, if the Euclidean distance between the defects generated on them exceeds the determined distance threshold, it is determined to be a violation of the component association constraint. The connection graph structure, propagation weight, spatial proximity requirements and constraint check rules are output as the component-level constraints of the generated model.

[0031] In a preferred embodiment of the present invention, step three, which involves encoding a regularized conditional vector embedding generator network and controlling the spatial location, morphology, and cross-component correlation of defects, includes:

[0032] Acquire 3D spatial coordinate data, component topology data, and co-occurrence probability matrix. Based on the spatial range of the forbidden area, generate a 3D spatial binary mask. Expand the co-occurrence probability matrix into a one-dimensional array in row-major order. Extract graph embedding vectors from the component topology map. Connect the spatial forbidden code, morphological co-occurrence code, and component topology code in sequence to generate a fixed-length conditional input vector.

[0033] The random noise vector is concatenated with the conditional input vector and used as the input to the generator. The input is fed into the generator, which outputs a defect distribution image in three-dimensional space. Before the generation result is output, the pixel values ​​of the corresponding positions of the forbidden areas in the defect distribution image are set to zero. All generated defect pairs are checked according to the co-occurrence judgment rule.

[0034] For a pair of defects on a connecting component, calculate their Euclidean distance. If the distance exceeds a predetermined distance threshold, move the position of one defect along the connecting line to the other defect until the distance equals the distance threshold. Output a defect image that completes the spatial position, shape, and cross-component correlation control.

[0035] In a preferred embodiment of the present invention, the process of extracting multi-scale features from the visible light image and the infrared thermal image in step four includes:

[0036] Acquire visible light image X V and infrared image X I As input data, use neural network F V and F I Visible light images and infrared images are processed separately. Visible light images are processed through a neural network F. V Feature extraction: Infrared images are processed by a neural network F. I Feature extraction;

[0037] Construct a feature pyramid and use the feature pyramid structure for cross-layer feature fusion. The specific steps are as follows:

[0038] S1: Starting from the top of the feature pyramid, proceed down to the bottom layer and perform feature fusion operation at each layer;

[0039] S2: For the l-th layer, first adjust the channel dimensions using a 1×1 convolution operation, so that... and The number of channels is consistent, and then a 3×3 convolution operation is applied to capture context information;

[0040] S3: Adjust the size using an upsampling operation, so that Size and Match the features, perform a fusion operation, and obtain the fused feature map F. l ;

[0041] Infrared feature map for each level A spatial attention mechanism is applied to generate a single-channel spatial attention map, and the generated spatial attention map is multiplied back into the original feature map element by element.

[0042] In a preferred embodiment of the present invention, the process of fusing cross-modal representations in step four includes:

[0043] Obtain infrared feature maps after channel dimension adjustment, feature enhancement, and spatial scale unification. Infrared feature maps after channel dimension adjustment, feature enhancement, and spatial scale unification processing By applying a spatial attention mechanism, the infrared feature map, after channel dimension adjustment, feature enhancement, spatial scale unification, and spatial attention, is stitched together with the corresponding visible light feature map along the channel dimension to form a cross-modal fusion feature map F. l ;

[0044] Using the generated cross-modal fusion feature map F l As input to the classifier and detector, a contrastive loss function is introduced during model training to measure the consistency of features between different modalities. In the deep modeling of multimodal interactions, the features of the two modalities are flattened into sequences, and the attention output is calculated based on the query, key, and value matrix.

[0045] In a preferred embodiment of the present invention, step four, which involves modeling environmental and equipment conditions and collaboratively generating paired visible light and infrared defect images via a generative network, includes:

[0046] Collect information about the current environment, encode this information into an environment condition vector, use an adaptive instance normalization method to inject the environment condition vector into the generative adversarial network, integrate the environment condition vector into the condition modulation mechanism parameters, design a shared encoder to extract high-level feature representations from the original input data, and perform fusion processing on the extracted features.

[0047] The fused features are fed into two independent decoders. Each decoder performs image reconstruction based on its assigned modality and modality-specific information. A discriminator evaluates the generated image, and the difference between the generated image and the real image is calculated using the L1 norm. The analysis is based on the activation values ​​of the intermediate layers of the pre-trained deep network.

[0048] Compared with the prior art, the advantages of this invention are:

[0049] (1) In this invention, prohibited areas, co-occurrence rules and component associations are constructed by integrating historical and equipment knowledge. Masks are generated by kernel density estimation to suppress invalid defects. Co-occurrence frequency is statistically analyzed based on feature vectors and co-occurrence relationship is determined by combining distance and similarity. A connection graph is established based on the topology. Weights and thresholds are set to constrain defect proximity. Masks, matrices and graphs are embedded and encoded as condition vectors. In the generation network, prohibited areas are set to zero, illegal co-occurrences are filtered, and the position of cross-component defects is adjusted to accurately control defect distribution and enhance physical rationality and engineering credibility.

[0050] (2) In this invention, visible light and infrared images are generated collaboratively by fusing multi-scale features, cross-modal stitching and environmental conditions. The dual-modal features are stitched together after pyramid and attention enhancement. Contrast loss is introduced to improve consistency. The environmental vector is injected through adaptive normalization. The shared encoder and dual decoder complete the reconstruction. The discriminator combines L1 and perceptual loss to ensure authenticity. The generation quality and practicality are improved in terms of alignment, rationality and consistency. Attached Figure Description

[0051] Figure 1 This is a flowchart of the inspection sample image data enhancement method for power artificial intelligence platform in this invention;

[0052] Figure 2 This is a flowchart illustrating the steps of cross-layer feature fusion using a feature pyramid structure in this invention. Detailed Implementation

[0053] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0054] Example 1: As Figure 1 As shown, the present invention proposes a method for enhancing inspection sample image data for power artificial intelligence platforms, comprising the following steps:

[0055] Step 1: Collect inspection image data, environmental monitoring data, and spatial positioning data generated during high-altitude inspections, close-range on-site inspections, and continuous monitoring of fixed areas of power equipment. Inspection image data includes visible light images and infrared thermal images; environmental monitoring data includes ambient temperature, ambient humidity, and light intensity; and spatial positioning data includes three-dimensional geographic coordinates and equipment attitude angles. Preprocess the collected data: denoise the inspection image data; clean and standardize the environmental monitoring and spatial positioning data; generate defect annotation data for typical defect areas in the images; align images, annotations, and non-image data using timestamps and spatial locations; and construct a structured initial sample set containing visual content, defect labels, and environmental and spatial conditions as the basic input for subsequent data augmentation.

[0056] By simultaneously acquiring inspection images, environmental monitoring, and spatial positioning data, multimodal fusion of visible light and infrared images is achieved. Preprocessing techniques such as denoising, cleaning, and standardization are used to ensure data quality. By generating defect annotations and aligning multi-source data based on timestamps and spatial locations, a structured initial sample set containing visual content, defect labels, and environmental spatial conditions is constructed. This not only improves the accuracy and consistency of the data but also provides a foundation for the model to learn the correlation between defect features and external conditions. It supports subsequent data augmentation and environmental adaptive detection tasks and has good integrity, scalability, and engineering application value.

[0057] Step 2: Using equipment type, ambient temperature, ambient humidity, light intensity, 3D geographic coordinates, and equipment attitude angle, combined with thermodynamic and optical response laws, a network is trained to synthesize images with defects. The network architecture adopts a conditional diffusion model as the backbone of the network, supporting multi-condition vector injection. A local heating model is established based on defect type and electrical load characteristics. Combining the emissivity of equipment material, ambient temperature, and heat exchange conditions, the radiation intensity is calculated using the Stefan-Boltzmann law. Through the temperature distribution and radiation intensity calculation results output by the local heating model, a physical perception infrared image generation mechanism is constructed.

[0058] Step two, which utilizes equipment type, ambient temperature, ambient humidity, light intensity, three-dimensional geographic coordinates, and equipment attitude angle, combined with thermodynamic and optical response laws, includes the following processes:

[0059] The system obtains device type, ambient temperature, ambient humidity, light intensity, 3D geographic coordinates, and device attitude angle as input parameters. The device type is extracted from the predefined category set and converted into a one-hot encoded vector. The vector length is equal to the total number of categories. The value at the corresponding position of the category is 1, and the value at other positions is 0. The one-hot encoded vector is mapped to a continuous real number vector of fixed dimension through the embedding layer. The vector dimension is a constant set before training. The ambient temperature and ambient humidity are standardized by using the mean and standard deviation recorded in the training dataset. The original value is subtracted from the mean and then divided by the standard deviation to obtain the standardized value. The light intensity is linearly normalized by using the minimum and maximum values ​​recorded in the training dataset. The current value is subtracted from the minimum value and then divided by the difference between the maximum and minimum values. The result is mapped to the interval between 0 and 1.

[0060] The device installation attitude angle is expressed in radians. The sine and cosine values ​​of the attitude angle are calculated to form a two-dimensional real vector. The embedded vector of the device type, the standardized ambient temperature, the standardized ambient humidity, the normalized light intensity, the sine value of the attitude angle, and the cosine value of the attitude angle are connected in sequence to generate a real vector of fixed length.

[0061] Calculate the heat generated by the conductive components. The heat generated is equal to the square of the current, the product of the resistance and the energizing time. When the equipment has defects such as poor contact, corrosion, or aging, the local resistance value is taken as an increased value. Calculate the heat transfer rate inside the equipment. The heat transfer rate is equal to the product of the material's thermal conductivity and the temperature gradient of the adjacent area. The material's thermal conductivity is taken from a fixed value in the material parameter table. The heat diffusion path is determined by the material distribution and geometry. Calculate the radiant energy density on the equipment surface. The radiant energy density is equal to the material emissivity, the Stefan-Boltzmann constant, and the surface temperature. The product of the fourth power of the degree, the material emissivity is taken from a fixed value in the material parameter table, the pixel brightness value of the infrared image is calculated, the pixel brightness value is equal to the product of the system response coefficient, the material emissivity, the Stefan-Boltzmann constant and the fourth power of the surface temperature, the system response coefficient is a fixed value set before training, the reflected energy of the device surface to the incident light is calculated, the incident light energy is equal to the sum of the reflected energy, absorbed energy and transmitted energy, the power equipment material is an opaque body, the transmitted energy is set to 0, the sum of the absorptivity and reflectivity is set to 1, and the absorbed light energy is included in the surface heat energy calculation;

[0062] The device type embedding vector, environmental parameters, attitude angle representation, heat generation, heat transfer rate, radiant energy density, pixel brightness value, and reflected energy are all used as input parameters and input into the corresponding module of the generation model.

[0063] Step two involves training the network to synthesize defective images, including:

[0064] Acquire a real infrared image, device type, ambient temperature, ambient humidity, light intensity, device attitude angle, and defect type. Perform a T-step noise addition operation on the real infrared image. In each step, determine the noise variance based on the value at the corresponding position of the noise scheduling sequence and add Gaussian noise to the image. The initial image is a real infrared image. After T steps, the image becomes a pure noise image with a standard normal distribution.

[0065] The defect type is taken from the predefined category set and converted into a one-hot encoded vector. The vector length is equal to the total number of defect categories. The one-hot encoded vector of the defect type, the embedded vector of the equipment type, the normalized ambient temperature, the normalized ambient humidity, the normalized light intensity, the sine value of the attitude angle, and the cosine value of the attitude angle are connected in sequence to generate a real number vector of fixed length.

[0066] At each time step of the denoising process, the noisy image, time step number, and extended conditional input vector are input into the generator network. The generator network operates in an encoder-decoder structure. In each layer, the extended conditional input vector information is fused through a cross-attention mechanism, and the output is an estimate of the noise added at the current time step. Using the noise estimate, the noisy image, and fixed coefficients in the noise scheduling sequence, the denoised image is calculated according to the denoising update rule. The update process does not introduce additional random variables. Starting from the last time step, the image state of each time step is calculated sequentially until all denoising steps are completed, and the denoised image is output.

[0067] The main loss term, physical consistency loss term, and perceptual loss term are calculated. The main loss term is calculated as the mean square error between the noise estimate output by the generator network and the actual noise added. The temperature value at the corresponding position is inferred from the brightness value of each pixel position in the generated image. The absolute difference between the inferred temperature value and the theoretical radiation intensity is calculated as the physical consistency loss term. The generated image and the corresponding real infrared image are input into the pre-trained VGG network to extract the feature map of the thirteenth layer. The Euclidean distance between the two feature maps is calculated as the perceptual loss term. The main loss term, physical consistency loss term, and perceptual loss term are weighted and summed with fixed weight coefficients, which are constants set before training. The total loss value is obtained. Based on the total loss value, the parameters of the generator network are updated using the gradient descent method.

[0068] By integrating device type, environmental parameters, and spatial attitude, a generative framework is constructed based on a conditional diffusion model. One-hot encoding, embedding mapping, standardization, and trigonometric transformation are used to achieve a unified representation and splicing of multimodal conditional vectors. Combining thermodynamic and optical mechanisms, a local heating model is established, using current, resistance, thermal conductivity, emissivity, and the Stefan-Boltzmann law to calculate temperature distribution, radiation intensity, and pixel brightness, considering the relationship between light reflection and heat absorption to improve the physical plausibility of the image. During denoising, conditional information is fused through an encoder-decoder structure and cross-attention, and deterministic image updates are used to reconstruct the image. Weighted optimization with principal loss, physical consistency loss, and perceptual loss ensures that the generated result approximates real data at the pixel, physical law, and visual perception levels, enhancing the realism, interpretability, and generalization ability of the synthesized image and providing high-quality, diverse defect sample support for model training.

[0069] Step 3: Based on the spatial distribution characteristics of defect annotations and the topological relationship of components, construct defect location forbidden areas, morphological co-occurrence rules and component association constraints, encode regularized condition vectors and embed them into the generative network to regulate the spatial location, morphology and cross-component association of defects;

[0070] Step three involves constructing the defect location forbidden zone and morphological co-occurrence rules, which includes:

[0071] Retrieve the spatial locations of all historical defect samples from the historical database, denoted as N. Set the bandwidth parameter h for kernel density estimation, with the parameter value being a constant manually set before training. Set the kernel function type to Gaussian kernel. For each calculation location x in three-dimensional space, calculate the defect generation probability density value at location x using the following formula:

[0072] Where p(x) is the probability density value at position x, x is the currently calculated 3D spatial position, N is the total number of historical defect samples, obtained from historical database statistics, h is the bandwidth parameter for kernel density estimation, manually set before training, and x i Let K(·) be the spatial location of the i-th historical defect sample, and K(·) be the kernel function, using a Gaussian kernel. This is the normalized distance vector between the current calculated position and the i-th sample point;

[0073] Read the geometric coordinates from the equipment design model, extract the spatial range of the key engineering area, mark all spatial points within the key engineering area, and perform corresponding operations to construct the forbidden area: mark all spatial locations that satisfy p(x)<τ as low probability areas, where τ is a constant threshold manually set before training, perform a union operation between the low probability areas and the key engineering area to generate the forbidden area, and set the defect generation probability density value of all spatial points within the forbidden area to zero before outputting the generation result;

[0074] For each type of defect, a four-dimensional feature vector is constructed. The vector components are: shape parameter, size, boundary curvature, and aspect ratio. The feature vectors of the five types of defects are stored in the feature database. Based on historical data, the following operations are performed: the frequency of all defect type pairs occurring simultaneously in neighboring locations is counted. Proximity is determined by the Euclidean distance being less than a distance threshold manually set before training. The frequency is normalized to co-occurrence probability and filled into a 5×5 symmetric matrix. The matrix is ​​initialized before training begins.

[0075] For any two defects A and B, use a function to calculate the co-occurrence determination result:

[0076]

[0077] Among them, Rule co (A,B) represents the co-occurrence determination result of defects A and B, where 1 indicates allowed, 0 indicates prohibited, and x A Let x be the spatial location of defect A. B Let ||x| be the spatial location of defect B. A -x B || represents the Euclidean distance between the two defects, δ is the spatial distance threshold, manually set before training, and f A Let f be the morphological feature vector of defect A. B Let f be the morphological feature vector of defect B, cos(f A ,f B P(t) represents the cosine similarity between two feature vectors. A ,t B ) represents defect type t A With t B In the co-occurrence probability matrix, ρ is the morphological similarity threshold, which is manually set before training, and θ is the co-occurrence probability threshold, which is manually set before training.

[0078] Output the definition of forbidden regions and the rules for determining the co-occurrence of defects as constraints for the generative model;

[0079] Step three involves constructing component association constraints, which includes:

[0080] The geometric coordinates of seven structural units are read from the equipment design model to determine the fixed spatial area of ​​each component. The components are numbered from 1 to 7. An undirected connection graph between the components is constructed according to the mechanical assembly relationship. If two components have a mechanical connection, an edge is added to the graph. The graph structure is initialized before training begins. The strength benchmark value of each connection type is read from the connection parameter table. The parameter values ​​are fixed by the system structure and do not change with the training process.

[0081] For each pair of connected components B i With B jPerform the following operations: count the number of times both components have defects simultaneously from historical data, query the strength benchmark value corresponding to the connection type of the two components, divide the count by the strength benchmark value to obtain the propagation weight, and assign all propagation weights before training begins.

[0082] Construct a static mapping table, where the input is the propagation weight and the output is the distance threshold. The mapping relationship is constructed before training begins and is not updated during the training process. For each edge in the graph (i.e. each pair of connected components), if its propagation weight is greater than zero, perform the following operations: query the mapping table to obtain the distance threshold corresponding to the propagation weight, and use the distance threshold as the spatial proximity requirement for defects on this pair of components.

[0083] For each pair of connected components, if the Euclidean distance between the defects generated on them exceeds the determined distance threshold, it is determined to be a violation of the component association constraint. The connection graph structure, propagation weight, spatial proximity requirements and constraint check rules are output as the component-level constraint conditions of the generated model.

[0084] Step three involves encoding regularization conditional vector embedding to generate a network and controlling the spatial location, morphology, and cross-component correlation of defects.

[0085] Acquire 3D spatial coordinate data, component topology data, and co-occurrence probability matrix. Based on the spatial range of the forbidden area, generate a 3D spatial binary mask. Set the voxel value within the forbidden area to 1 and the voxel value in other areas to 0. Expand the co-occurrence probability matrix into a one-dimensional array with a length of 25 in row-major order. Extract the graph embedding vector from the component topology diagram with a vector dimension of 16. Connect the spatial forbidden code, morphological co-occurrence code, and component topology code in sequence to generate a fixed-length conditional input vector.

[0086] The random noise vector is concatenated with the conditional input vector and used as the input to the generator. The input is fed into the generator, which outputs a defect distribution image in three-dimensional space. Before the generation result is output, the pixel values ​​of the corresponding positions of the forbidden areas in the defect distribution image are set to zero. All generated defect pairs are checked according to the co-occurrence judgment rule. If the judgment result is 0, the pixel values ​​of the related defects in the defect pair are set to zero.

[0087] For a pair of defects on a connecting component, calculate their Euclidean distance. If the distance exceeds a predetermined distance threshold, move the position of one defect along the connecting line to the other defect until the distance equals the distance threshold. Output a defect image that completes the spatial position, shape and cross-component correlation control.

[0088] By integrating historical defect distribution with prior equipment knowledge, spatial forbidden zones, morphological co-occurrence rules, and component association constraints are constructed. Kernel density estimation combined with key regions is used to generate forbidden masks to suppress the generation of invalid defects. Based on features such as shape and size, a four-dimensional vector is constructed, and the co-occurrence frequency is statistically analyzed to form a probability matrix. Co-occurrence relationships are determined by combining spatial distance and morphological similarity. A connection graph is established based on the equipment topology, and propagation weights and distance thresholds are introduced to constrain the spatial proximity of defects in connected components. The forbidden mask, co-occurrence matrix, and graph embedding are encoded into conditional vectors and embedded into the generation network. During generation, forbidden zones are zeroed out, illegal co-occurrences are filtered out, and the positions of cross-component defects are adjusted to meet the constraints. This achieves precise control over defect position, morphology, and cross-component association, improving the physical rationality and engineering credibility of the generated results.

[0089] Example 2: The technical solution of this embodiment of the invention differs from that of Example 1 in that:

[0090] like Figure 2 As shown, step four: extract multi-scale features from visible light images and infrared thermal images, fuse cross-modal representations, and combine device type, ambient temperature, ambient humidity, light intensity and component-level semantic constraints to generate paired visible light and infrared defect images through a generative network.

[0091] Step four involves extracting multi-scale features from visible light images and infrared thermal images, including:

[0092] Acquire visible light image X V and infrared image X I As input data, use neural network F V and F I Visible light images and infrared images are processed separately. Visible light images are processed through a neural network F. V Extract features and output the feature map set {f} for each layer. V1 ,f V2 ,…,f Vl}, where f Vl This represents the visible light feature map of the l-th layer. The infrared image is processed by the neural network F. I Extract features and output the feature map set {f} for each layer. I1 ,f I2 ,…,f Il}, where f Il This represents the infrared feature map of the l-th layer, and two neural networks F... V and F I The structures are identical, but the weights are trained independently.

[0093] Construct a feature pyramid and use the feature pyramid structure for cross-layer feature fusion. The specific steps are as follows:

[0094] S1: Starting from the top of the feature pyramid, proceed down to the bottom layer and perform feature fusion operation at each layer;

[0095] S2: For the l-th layer, first adjust the channel dimensions using a 1×1 convolution operation, so that... and The number of channels is consistent, and then a 3×3 convolution operation is applied to capture context information;

[0096] S3: Adjust the size using an upsampling operation, so that Size and Match and perform the merge operation:

[0097] F l =f Vl +UpSample(Conv 3×3 (Conv 1×1 (f Il ))), where F l f is the feature map after fusion at layer l. Vl f is the feature map extracted from the visible light image at layer l. Il Conv is the feature map extracted from the infrared image at layer l. 1×1 This is a 1x1 convolution operation used for channel-dimensional adjustments (i.e., channel alignment) to facilitate the fusion of features from different modalities. 3×3 The 3x3 convolution operation is used to capture more contextual information and help enhance feature representation. UpSample is an upsampling operation used to match the feature map size to obtain the fused feature map F. l ;

[0098] Infrared feature map for each level Spatial attention mechanisms are applied, including global average pooling, global max pooling, channel concatenation, and convolution operations, to generate a single-channel spatial attention map. The generated spatial attention map is then multiplied back into the original feature map element by element, thereby enhancing the information representation of important regions.

[0099] Step four involves the process of fusing cross-modal representations, which includes:

[0100] Obtain infrared feature maps after channel dimension adjustment, feature enhancement, and spatial scale unification. Infrared feature maps after channel dimension adjustment, feature enhancement, and spatial scale unification processing By applying a spatial attention mechanism, the infrared feature map, after channel dimension adjustment, feature enhancement, spatial scale unification, and spatial attention, is stitched together with the corresponding visible light feature map along the channel dimension to form a cross-modal fusion feature map F. l Using the generated cross-modal fusion feature map F lAs input to the classifier and detector, a contrastive loss function is introduced during model training to measure the consistency of features between different modalities. The contrastive loss function is expressed by the following formula:

[0101] Where L contrastive To compare loss values, z is used to measure the consistency of features learned by the model across different modalities. v Let z be the feature representation vector of the visible light image. i Let be the feature representation vector of the infrared image, sim(·,·) be the similarity function, and cosine similarity is usually used to measure the similarity between two feature vectors. τ is a temperature parameter used to control the sharpness of the distribution.

[0102] In the deep modeling of multimodal interactions, the features of the two modalities are flattened into sequences, and attention output is calculated based on query, key, and value matrices to capture the interaction between the two modalities.

[0103] Step four involves modeling environmental and equipment conditions and collaboratively generating paired visible and infrared defect images using a generative network.

[0104] Collect information about the current environment (temperature, humidity, light intensity), encode this information into an environmental condition vector, use an adaptive instance normalization method to inject the environmental condition vector into the generative adversarial network, integrate the environmental condition vector into the conditional modulation mechanism parameters, design a shared encoder to extract high-level feature representations from the original input data (visible light and infrared images), and perform fusion processing on the extracted features;

[0105] After fusion processing, the features are fed into two independent decoders. The decoders reconstruct the image based on their respective modalities (visible light or infrared) and modality-specific information. The discriminator evaluates the generated image, and the L1 norm is used to calculate the difference between the generated image and the real image. The analysis is based on the activation values ​​of the intermediate layers of the pre-trained deep network.

[0106] By integrating multi-scale feature extraction, cross-modal fusion, and environmental condition modeling, this method achieves the collaborative generation of visible light and infrared defect images. It extracts multi-layer features from both modes using neural networks, and enhances contextual representation and key region response by combining feature pyramids and spatial attention mechanisms. The processed infrared and visible light features are concatenated along the channel dimension to form a unified cross-modal representation for subsequent tasks, and a contrastive loss is introduced to improve intermodal consistency. Conditional vectors are constructed using environmental parameters and injected into the generation network through adaptive instance normalization. A shared encoder and dual-decoder architecture are combined to achieve modality-specific reconstruction. The generated results are evaluated by a discriminator, and L1 loss and perceptual analysis are used to ensure image authenticity. The overall method demonstrates outstanding performance in feature alignment, physical plausibility, and cross-modal consistency, improving the accuracy and engineering applicability of defect image generation.

[0107] The above are merely preferred embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and its improved concept, should be covered within the scope of protection of the present invention.

Claims

1. A method for enhancing inspection sample image data for a power artificial intelligence platform, characterized in that, Includes the following steps: Step 1: Collect inspection image data, environmental monitoring data, and spatial positioning data generated during high-altitude inspection, on-site close-range inspection, and continuous monitoring of fixed areas of power equipment. Preprocess the collected data to generate defect labeling data for typical defect areas in the images. Align the images, labels, and non-image data through timestamps and spatial locations to construct a structured initial sample set containing visual content, defect labels, and environmental and spatial conditions. Step 2: Using equipment type, ambient temperature, ambient humidity, light intensity, three-dimensional geographic coordinates, and equipment attitude angle, combined with thermodynamic and optical response laws, train and generate a network to synthesize defective images; Step 3: Based on the spatial distribution characteristics of defect annotations and the topological relationship of components, construct defect location forbidden areas, morphological co-occurrence rules and component association constraints, encode regularized condition vectors and embed them into the generative network to regulate the spatial location, morphology and cross-component association of defects; Step 4: Extract multi-scale features from visible light images and infrared thermal images, fuse cross-modal representations, and combine device type, ambient temperature, ambient humidity, light intensity, and component-level semantic constraints to generate paired visible light and infrared defect images through a generative network.

2. The method for enhancing inspection sample image data for a power artificial intelligence platform according to claim 1, characterized in that, Step two, which utilizes equipment type, ambient temperature, ambient humidity, light intensity, three-dimensional geographic coordinates, and equipment attitude angle, combined with thermodynamic and optical response laws, includes the following: The device type, ambient temperature, ambient humidity, light intensity, 3D geographic coordinates, and device attitude angle are obtained as input parameters. The device type is extracted from the predefined category set and converted into a one-hot encoded vector with a length equal to the total number of categories. The one-hot encoded vector is mapped to a continuous real number vector of fixed dimension through an embedding layer. The ambient temperature and ambient humidity are standardized, and the light intensity is linearly normalized. The device installation attitude angle is expressed in radians. The sine and cosine values ​​of the attitude angle are calculated to form a two-dimensional real vector. The embedded vector of the device type, the standardized ambient temperature, the standardized ambient humidity, the normalized light intensity, the sine value of the attitude angle, and the cosine value of the attitude angle are connected in sequence to generate a real vector of fixed length. Calculate the heat generated by conductive components, calculate the heat transfer rate inside the device, calculate the radiant energy density on the surface of the device, calculate the pixel brightness value of the infrared image, and calculate the energy reflected by the device surface from incident light. The device type embedding vector, environmental parameters, attitude angle representation, heat generation, heat transfer rate, radiative energy density, pixel brightness value, and reflected energy are all used as input parameters and input into the corresponding module of the generation model.

3. The method for enhancing inspection sample image data for a power artificial intelligence platform according to claim 2, characterized in that, The process of training the network to synthesize defective images in step two includes: Acquire a real infrared image, device type, ambient temperature, ambient humidity, light intensity, device attitude angle, and defect type. Perform a T-step noise addition operation on the real infrared image. After T steps, the image becomes a pure noise image with a standard normal distribution. The defect type is taken from the predefined category set and converted into a one-hot encoded vector. The vector length is equal to the total number of defect categories. The one-hot encoded vector of the defect type, the embedded vector of the equipment type, the normalized ambient temperature, the normalized ambient humidity, the normalized light intensity, the sine value of the attitude angle, and the cosine value of the attitude angle are connected in sequence to generate a real number vector of fixed length. At each time step of the denoising process, the noisy image, time step number, and extended conditional input vector are input into the generator network. The generator network operates in an encoder-decoder structure. In each layer, the extended conditional input vector information is fused through a cross-attention mechanism, and the output is an estimate of the noise added at the current time step. Using the noise estimate, the noisy image, and fixed coefficients in the noise scheduling sequence, the denoised image is calculated according to the denoising update rule, and the denoised image is output. The main loss term, physical consistency loss term, and perceptual loss term are calculated. The main loss term, physical consistency loss term, and perceptual loss term are then summed by weighting them with fixed weight coefficients, which are constants set before training, to obtain the total loss value. Based on the total loss value, the parameters of the generator network are updated using gradient descent.

4. The method for enhancing inspection sample image data for a power artificial intelligence platform according to claim 1, characterized in that, The process of constructing defect location forbidden regions and morphological co-occurrence rules in step three includes: Read the spatial locations of all historical defect samples from the historical database, set the bandwidth parameter h for kernel density estimation, and set the parameter value as a constant manually set before training. For each calculation location x in three-dimensional space, use the formula to calculate the defect generation probability density value at location x. Read the geometric coordinates from the equipment design model, extract the spatial range of the key engineering area, mark all spatial points within the key engineering area, perform corresponding operations to construct the restricted area, and set the defect generation probability density value of all spatial points within the restricted area to zero before generating the output results. For each type of defect, a four-dimensional feature vector is constructed, and corresponding processing operations are performed based on historical data. For any two defects A and B, the co-occurrence determination result is calculated using a function, and the forbidden region definition and defect co-occurrence determination rules are output as constraints for the generative model.

5. The method for enhancing inspection sample image data for a power artificial intelligence platform according to claim 4, characterized in that, The process of constructing component association constraints in step three includes: Read the geometric coordinates of seven structural units from the equipment design model, determine the fixed spatial area of ​​each component, construct an undirected connection graph between components based on the mechanical assembly relationship, add an edge to the graph if two components have a mechanical connection, and read the strength reference value of each connection type from the connection parameter table. For each pair of connected components B i With B j Perform the corresponding processing operations, construct a static mapping table, with the propagation weight as the input and the distance threshold as the output. For each edge in the graph, if its propagation weight is greater than zero, then perform the corresponding processing operations. For each pair of connected components, if the Euclidean distance between the defects generated on them exceeds the determined distance threshold, it is determined to be a violation of the component association constraint. The connection graph structure, propagation weight, spatial proximity requirements and constraint check rules are output as the component-level constraints of the generated model.

6. The method for enhancing inspection sample image data for a power artificial intelligence platform according to claim 5, characterized in that, The process of encoding regularized conditional vector embedding to generate a network and controlling the spatial location, morphology, and cross-component correlation of defects in step three includes: Acquire 3D spatial coordinate data, component topology data, and co-occurrence probability matrix. Based on the spatial range of the forbidden area, generate a 3D spatial binary mask. Expand the co-occurrence probability matrix into a one-dimensional array in row-major order. Extract graph embedding vectors from the component topology map. Connect the spatial forbidden code, morphological co-occurrence code, and component topology code in sequence to generate a fixed-length conditional input vector. The random noise vector is concatenated with the conditional input vector and used as the input to the generator. The input is fed into the generator, which outputs a defect distribution image in three-dimensional space. Before the generation result is output, the pixel values ​​of the corresponding positions of the forbidden areas in the defect distribution image are set to zero. All generated defect pairs are checked according to the co-occurrence judgment rule. For a pair of defects on a connecting component, calculate their Euclidean distance. If the distance exceeds a predetermined distance threshold, move the position of one defect along the connecting line to the other defect until the distance equals the distance threshold. Output a defect image that completes the spatial position, shape, and cross-component correlation control.

7. The method for enhancing inspection sample image data for a power artificial intelligence platform according to claim 1, characterized in that, The process of extracting multi-scale features from visible light images and infrared thermal images in step four includes: Acquire visible light image X V and infrared image X I As input data, use neural network F V and F I Visible light images and infrared images are processed separately. Visible light images are processed through a neural network F. V Feature extraction: Infrared images are processed by a neural network F. I Feature extraction; Construct a feature pyramid and use the feature pyramid structure for cross-layer feature fusion. The specific steps are as follows: S1: Starting from the top of the feature pyramid, proceed down to the bottom layer and perform feature fusion operation at each layer; S2: For the l-th layer, first adjust the channel dimensions using a 1×1 convolution operation, so that... and The number of channels is consistent, and then a 3×3 convolution operation is applied to capture context information; S3: Adjust the size using an upsampling operation, so that Size and Match the features, perform a fusion operation, and obtain the fused feature map F. l ; Infrared feature map for each level A spatial attention mechanism is applied to generate a single-channel spatial attention map, and the generated spatial attention map is multiplied back into the original feature map element by element.

8. The method for enhancing inspection sample image data for a power artificial intelligence platform according to claim 7, characterized in that, The process of fusing cross-modal representations in step four includes: Obtain infrared feature maps after channel dimension adjustment, feature enhancement, and spatial scale unification. Infrared feature maps after channel dimension adjustment, feature enhancement, and spatial scale unification processing By applying a spatial attention mechanism, the infrared feature map, after channel dimension adjustment, feature enhancement, spatial scale unification, and spatial attention, is stitched together with the corresponding visible light feature map along the channel dimension to form a cross-modal fusion feature map F. l ; Using the generated cross-modal fusion feature map F l As input to the classifier and detector, a contrastive loss function is introduced during model training to measure the consistency of features between different modalities. In the deep modeling of multimodal interactions, the features of the two modalities are flattened into sequences, and the attention output is calculated based on the query, key, and value matrix.

9. A method for enhancing inspection sample image data for a power artificial intelligence platform according to claim 8, characterized in that, The process of modeling environmental and equipment conditions and collaboratively generating paired visible light and infrared defect images through a generative network in step four includes: Collect information about the current environment, encode this information into an environment condition vector, use an adaptive instance normalization method to inject the environment condition vector into the generative adversarial network, integrate the environment condition vector into the condition modulation mechanism parameters, design a shared encoder to extract high-level feature representations from the original input data, and perform fusion processing on the extracted features. The fused features are fed into two independent decoders. Each decoder performs image reconstruction based on its assigned modality and modality-specific information. A discriminator evaluates the generated image, and the difference between the generated image and the real image is calculated using the L1 norm. The analysis is based on the activation values ​​of the intermediate layers of the pre-trained deep network.

Images