Large model-based power image sample data processing method and device, and storage medium
By adopting large-model technology in power image analysis, combining convolutional neural networks and Bayesian neural networks to process noise, and using generative adversarial networks to generate samples, the problem of large-scale impact on power image annotation and insufficient sample number is solved, and more efficient and accurate power image analysis is achieved.
Patent Information
- Application Number
- CN202411626179.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-14
- Publication Date
- 2025-03-07
AI Technical Summary
Power image labeling is greatly affected by noise and insufficient sample size.
The power image sample data processing method based on large models is used to extract features through convolutional neural networks, combine with Bayesian neural networks to process noise, generate new samples using the generative adversarial network, and evaluate the similarity and distribution consistency of the generated samples through average absolute error and Fréchet distance.
It effectively reduces the uncertainty of labeling, expands the training set, improves the generalization ability and performance of the model, and makes power image analysis more accurate and efficient.
Smart Images
Figure CN119580031A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of power image data processing, and in particular to a method, device and storage medium for processing power image sample data based on a large model. Background Art
[0002] With the rapid development of power systems and the application of information technology, the demand for processing and application of power image data is increasing.
[0003] In the early research of image analysis, traditional machine learning algorithms were the mainstream, including support vector machines (SVM), decision trees, random forests, K-nearest neighbors (KNN), and naive Bayes algorithms. These methods have achieved certain success in tasks such as image classification and target detection, but they have obvious limitations. The most prominent challenge is that these algorithms are highly dependent on manual feature extraction, that is, features (such as texture, color, shape, etc.) are manually designed by domain experts or researchers for use in the model.
[0004] With the rapid development of deep learning technology, especially the introduction of convolutional neural networks (CNN), the field of image analysis has undergone revolutionary changes. CNN is a deep neural network that can automatically learn features directly from raw images. It gradually extracts local and global information of the image through a series of convolutional layers to generate more abstract feature representations.
[0005] Based on deep learning technology, the rise of large models in recent years has further promoted the progress in the field of image analysis. Large models usually refer to deep learning models trained based on large-scale data sets and complex neural network structures, such as Transformers and more complex architectures such as GPT and ViT (Vision Transformers). However, the introduction of large models also brings new challenges. Due to their highly complex network structure and a large number of parameters, training large models requires huge computing resources, including high-performance GPU or TPU clusters. In addition, the model takes a long time to train, consumes high energy, and the demand for storage has also increased significantly. This makes how to strike a balance between performance improvement and computing resources a key issue in the current field of image analysis.
[0006] In general, conducting key technology research on the application of large models in power image analysis is of great significance for improving productivity, optimizing operation management and promoting technological innovation. Summary of the invention
[0007] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and to provide a method, device and storage medium for processing power image sample data based on a large model, so as to solve or partially solve the problem that power image annotation is greatly affected by noise and the number of samples is insufficient.
[0008] The purpose of the present invention can be achieved by the following technical solutions:
[0009] One aspect of the present invention provides a method for processing power image sample data based on a large model, comprising the following steps:
[0010] Obtaining power images, using convolutional neural networks to extract low-level features and high-level features as image features, using the image features and pre-acquired power field knowledge base information as inputs of a pre-trained large model, generating label information, obtaining real samples, and realizing the annotation of power images;
[0011] In the labeling process, Bayesian neural network is used to introduce probability distribution to maximize log-likelihood estimation and reduce labeling uncertainty.
[0012] The goal is to train a generative adversarial network so that the discriminator cannot distinguish between generated samples and real samples. New power images are generated using the trained generative adversarial network, and generated samples are obtained through annotation.
[0013] The mean absolute error and Fréchet distance are combined to evaluate the similarity and distribution consistency between real samples and generated samples.
[0014] As a preferred technical solution, the method of reducing the uncertainty of annotation by maximizing log-likelihood estimation is implemented by the following formula:
[0015]
[0016] Among them, I is the power image of the model input data, is the predicted output of the model, θ is the model parameter, are the optimized model parameters, Given the input data I and parameter θ, the model output is The conditional probability of .
[0017] As a preferred technical solution, the loss function of the generative adversarial network is:
[0018] min max L(G,D)=E Ireal～pdata(I) [log D(I real )]+E z～pz(z) [log(1-D(G(z)))]
[0019] Among them, L(G,D) is the loss function of the generative adversarial network, G and D are the generator and discriminator respectively, and I real is the real image, p data (I) is the distribution of real data, p z (z) is the noise distribution of the generator input, z is the random noise input, G(z) is the new power image generated by the generator, D(I real ) is the discriminator for the real sample I real The output of D(G(z)) is the output of the discriminator for the generated sample G(z). E[·] is the expected value symbol, E Ireal～pdata(I) [log D(I real )] for real data sample I real The expected loss is calculated, E z～pz(z) [log(1-D(G(z)))] is the expected calculation of the loss of generating sample G(z).
[0020] As a preferred technical solution, the mean absolute error and Fréchet distance are calculated using the following formula:
[0021]
[0022] FID=∥μ r -μ g ∥ 2 +Tr(Σ r +Σ g -2(Σ r Σ g ) 1 / 2 )
[0023] Among them, MAE and FID are mean absolute error and Fréchet distance respectively, G(z) is the generated sample, I real is the real sample, N is the total number of pixels in the sample, μ r and μ g are the feature means of the real samples and the generated samples, Σ r and Σ g is the corresponding covariance matrix, and Tr() represents the trace operator of the matrix.
[0024] As a preferred technical solution, the image features are obtained using the following formula:
[0025] F=CNN(I)
[0026] Among them, F is the feature vector of image features, CNN is the convolutional neural network, and I is the power image.
[0027] As a preferred technical solution, the tag information is obtained using the following formula:
[0028] L=NLP(F,K)
[0029] Among them, L is the label information, NLP is the large model, F is the feature vector of image features, and K is the knowledge base information in the power field.
[0030] As a preferred technical solution, the label information includes defects, equipment abnormalities, equipment overheating, line breaks, insulation aging and normal operation.
[0031] As a preferred technical solution, the large model is a BERT model or a GPT model.
[0032] Another aspect of the present invention provides an electronic device, comprising: one or more processors and a memory, wherein the memory stores one or more programs, and the one or more programs include instructions for executing the aforementioned large model-based power image sample data processing method.
[0033] Another aspect of the present invention provides a computer-readable storage medium comprising one or more programs for execution by one or more processors of an electronic device, wherein the one or more programs include instructions for executing the aforementioned large model-based power image sample data processing method.
[0034] Compared with the prior art, the present invention has at least one of the following beneficial effects:
[0035] (1) Fully consider the annotation of power images in noisy conditions: In the annotation process of samples containing noisy images, the present invention uses a Bayesian neural network (BNN) to deal with noise and uncertainty. Experimental results show that in the presence of noisy images, the performance of the BNN model is better than that of the traditional deep learning model.
[0036] (2) Expanding the sample set: The present invention takes into account the situation that the amount of power image sample data is insufficient. The sample generation technology based on the large model can generate new generated samples by learning the existing sample data, thereby expanding the training set.
[0037] (3) Accurately evaluate the similarity between generated samples and real samples: The present invention evaluates the distribution consistency and similarity between real samples and similarity samples from two aspects: mean absolute error and Fréchet distance, so as to accurately evaluate the quality of generated samples.
[0038] (4) Improve model performance and data utilization efficiency. By combining intelligent labeling and sample generation technology, the training data set can be effectively expanded and higher-quality and diversified data can be provided for the power image analysis model. This method not only speeds up the sample labeling, but also improves the generalization ability and performance of the model, making image analysis in the power system more accurate and efficient. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] Figure 1 Schematic diagram of a method for processing power image sample data based on a large model in an embodiment. DETAILED DESCRIPTION
[0040] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.
[0041] Example 1
[0042] In order to solve the problem of intelligent labeling of power image samples and generate high-quality training data, this embodiment provides a method for processing power image sample data based on a large model, see Figure 1 , the method comprises the following steps:
[0043] Step S1, intelligent labeling of power image samples.
[0044] First, use a deep learning model (such as a large-scale pre-trained model) to automatically learn the features of power images. This includes tasks such as feature extraction, image classification, and semantic segmentation. The specific steps are as follows:
[0045] Step S101, feature learning: The model extracts low-level and high-level features of the power image through a convolutional neural network (CNN). Assuming the input image is I, the model extracts its features through a series of convolutional layers and pooling layers: F = CNN (I), where F is a feature vector. Specifically, low-level features include edge features, corner features, texture features, and color features, and high-level features include shape features, object detection and recognition, semantic segmentation, and image description.
[0046] Step S102, annotation generation: By combining domain knowledge with natural language processing (NLP), the model automatically generates labels for the extracted image features. Use a pre-trained large model (such as BERT or GPT) to process text data and domain knowledge, and map features to categories or attribute labels. Assuming that the domain knowledge base is K, by combining the image feature F and the knowledge base information K, the model generates a label L: L = NLP (F, K), which may include categories such as "defect" and "equipment abnormality".
[0047] Step S2, dealing with noise and uncertainty.
[0048] In the annotation process, it is key to deal with noise and uncertainty in samples. This embodiment uses a Bayesian deep learning method to measure the uncertainty of annotation and filter out noise data. Bayesian neural networks process noisy data by introducing probability distributions to model parameters. Assume that the output of the model is P(y'|I,θ), where θ is the uncertainty representation of the model parameters. The model reduces uncertainty by maximizing the log-likelihood estimate: θ'=arg max(log P(y'|I,θ)), which helps to improve the accuracy and consistency of the annotation results.
[0049] Step S3: sample generation and training set expansion.
[0050] In order to solve the problem of insufficient power image sample data, a generative adversarial network (GAN) is designed and trained to generate new synthetic samples and enrich the training set.
[0051] Step S301, Generative Adversarial Network (GAN) design:
[0052] GAN consists of a generator network G and a discriminator network D. The generator G receives random noise z as input and generates a new power image G(z), while the discriminator D determines whether the input image is a real image I real Still generate image G(z).
[0053] The goal of the generator is to generate samples that are as realistic as possible, so that the discriminator cannot distinguish between generated samples and real samples. The loss function can be expressed as:
[0054] min max L(G,D)=E Ireal～pdata(I) [log D(I real )]+E z～pz(z) [log(1-D(G(z)))]
[0055] Among them, p data (I) is the distribution of real data, p z (z) is the noise distribution of the generator input.
[0056] Step S302, sample generation: During the training process, the generator generates a synthetic power image G, and the authenticity of the generated sample is evaluated by the discriminator D. The training process continuously optimizes the generator and the discriminator, and finally the generator can generate realistic power images.
[0057] Step S4, generating a similarity evaluation between the sample and the real sample.
[0058] In order to verify the similarity and distribution consistency between the generated synthetic samples and the real samples, multiple quantitative indicators are combined for evaluation.
[0059] (1) Mean absolute error (MAE): evaluates the difference between the generated sample G(z) and the real sample I real The pixel-level difference between them can be calculated by the mean absolute error (MAE): MAE = 1 / N∑|I i real -G(z) i |, where N is the total number of pixels in the sample.
[0060] (2) Fréchet Distance (FID): Fréchet Inception Distance (FID) is an important indicator to measure the similarity between the generated sample distribution and the real sample distribution. By calculating the distribution difference between the generated image and the real image in the high-dimensional feature space, FID can be expressed as: FID = ∥μ r -μ g ∥ 2 +Tr(Σ r +Σ g -2(Σ r Σ g ) 1 / 2 ), where μ r and μ g are the feature means of the real samples and the generated samples, Σ r and Σ g is their covariance matrix.
[0061] Step S5, improving model performance and data utilization efficiency.
[0062] By combining the above intelligent labeling and sample generation technology, the training data set is effectively expanded and higher quality and diversified data is provided for the power image analysis model. This method not only speeds up the sample labeling, but also improves the generalization ability and performance of the model, making image analysis in the power system more accurate and efficient.
[0063] The following is a description of specific power image data samples and the corresponding experimental results, which demonstrate the effectiveness of this method in intelligent labeling, noise processing, sample generation, and model performance improvement.
[0064] (1) Data sample description.
[0065] 1) Dataset description.
[0066] The experiment used a real power equipment image dataset, which covers image samples of various power equipment and their operating status. The specific information of the dataset is as follows:
[0067] Equipment type: transformer, distribution cabinet, cable, etc.
[0068] Image resolution: 256 × 256 pixels.
[0069] Total data samples: 5,000.
[0070] Image status classification:
[0071] -Normal device images: 3000 images.
[0072] - Faulty equipment images: 1500 images.
[0073] - Noise images: 500 images (including impurities, overexposure, light and shadow interference in the environment around the device, etc.).
[0074] Fault type marking: overheating, insulation aging, line breakage, equipment deformation, etc.
[0075] The dataset is divided into training and testing sets:
[0076] Training set: 4000 images (including 2400 normal device images, 1200 fault images, and 400 noise images).
[0077] Test set: 1000 images (600 normal device images, 300 fault images, and 100 noise images).
[0078] 2) Sample annotation examples
[0079] The annotation results of some image samples are as follows:
[0080] Image sample 001: Power distribution cabinet, labeled "Equipment overheating".
[0081] Image Sample 002: Electrical cable, labeled "Line break".
[0082] Image Sample 003: Transformer, labeled "Insulation Aging".
[0083] Image Sample 004: Normal transformer, labeled “Normal Operation”.
[0084] These annotations are automatically generated by the large model through computer vision and domain knowledge base, ensuring the accuracy of the annotations.
[0085] (2) Experimental results
[0086] 1) Intelligent labeling performance evaluation.
[0087] The experiment first evaluated the accuracy and efficiency of intelligent labeling of power image samples. The experiment was verified on 1,000 test set samples, and the results of automatic labeling and manual labeling were compared to obtain the following evaluation indicators:
[0088] Labeling accuracy: 92.3%
[0089] Annotation recall: 90.8%
[0090] F1 score: 91.5%
[0091] Comparison of annotation time: Automatic annotation takes an average of 1.2 seconds per image, while manual annotation takes about 20 seconds per image, which is about 16.7 times more efficient.
[0092] These results show that the large model-based intelligent annotation system has high accuracy in power image annotation and significantly speeds up the annotation process.
[0093] 2) Noise processing effect.
[0094] In samples containing noisy images, Bayesian neural network (BNN) is used to deal with noise and uncertainty. Experimental results show that in the presence of noisy images, the performance of the BNN model is better than that of the traditional deep learning model. The specific performance is:
[0095] Model accuracy without noise treatment: 84.7%.
[0096] Model accuracy after processing noise: 89.2%.
[0097] BNN effectively reduces the risk of misclassification when processing noisy data and significantly improves the robustness of the model.
[0098] 3) Sample generation and training set expansion.
[0099] In order to expand the dataset, a generative adversarial network (GAN) was used to generate synthetic power image samples. The experiment verified the effectiveness of the generated samples through the following steps:
[0100] Use the generator G(z) to generate new images of power equipment and add them to the training set.
[0101] The training set was expanded from the original 4,000 to 6,000, of which 2,000 were generated samples.
[0102] Retrain the image recognition model on the augmented training set and evaluate it on the test set.
[0103] The improvement of model performance after training set expansion is as follows:
[0104] Original training set model accuracy: 89.0%;
[0105] The accuracy of the model after the expansion training set is 92.5%;
[0106] Model recall on original training set: 87.6%;
[0107] The recall rate of the augmented training set model is 91.1%.
[0108] Through sample generation, the diversity and scale of the training set are effectively expanded, thereby improving the model's recognition ability in actual power images.
[0109] 4) Evaluate the similarity between generated samples and real samples.
[0110] In order to evaluate the authenticity of the generated samples, Fréchet Inception Distance (FID) is used to evaluate the distribution consistency between the generated images and the real images. The experimental results show:
[0111] Original FID score (without augmented samples): 45.7;
[0112] FID score after expansion (GAN generated samples): 28.3.
[0113] A lower FID score indicates that the generated image is closer to the real image in terms of feature distribution, further verifying the quality of the generated samples.
[0114] 5) Improved model performance.
[0115] Through the above intelligent labeling, noise processing and sample generation technologies, the overall performance of the model has been significantly improved. The final performance of the model in the power equipment image analysis task is summarized as follows:
[0116] Model accuracy after automatic annotation: 92.3%;
[0117] Model accuracy after noise processing: 89.2%;
[0118] Model accuracy after sample expansion: 92.5%;
[0119] The distribution consistency (FID) between generated samples and real samples is 28.3.
[0120] These experimental results show that the present invention can effectively improve the data quality of power image samples and enhance the performance and robustness of the image analysis model.
[0121] (3) Summary of implementation effects.
[0122] Intelligent labeling speeds up image labeling efficiency, significantly reduces manual intervention time, and ensures high accuracy of labeling results.
[0123] To deal with noise and uncertainty, by introducing Bayesian neural network, the accuracy of the model in dealing with noisy samples is greatly improved.
[0124] The sample generation technology successfully expanded the size of the training set, and the model performance was significantly improved on the expanded data set.
[0125] The high similarity between the generated samples and the real samples further verifies the effectiveness of GAN technology in sample generation and provides rich training data for the power image analysis model.
[0126] In summary, this method first uses large model technology and combines relevant theories in the fields of computer vision and natural language processing to realize intelligent labeling of power image samples. This includes learning the features of power images through large models, and automatically labeling relevant attributes and category information for image samples by associating images and domain knowledge. At the same time, it is also possible to study how to deal with noise and uncertainty in samples to improve the accuracy and consistency of labeling results. Secondly, a large amount of power image sample data is collected and divided into training sets and test sets. Next, we design and train a generative adversarial network (GAN) to learn the feature distribution of power images. During the training process, the generator network generates new power image samples, while the discriminator network is used to evaluate the authenticity of these samples. Subsequently, the similarity and distribution consistency between the generated samples and the real samples are quantitatively evaluated to verify the effect and feasibility of the generated model. By studying the application of large model technology in the processing of power image sample data, the utilization efficiency of large-scale power image sample data can be effectively improved, and high-quality and diverse training data can be provided for subsequent image recognition model research.
[0127] This method significantly improves the efficiency and accuracy of image analysis by using a large deep learning model combined with intelligent annotation and sample generation technology of power image sample data. The model can automatically learn the key features in power images and annotate the images, including features such as defects and anomalies, thereby speeding up sample annotation. When the amount of power image sample data is insufficient, the sample generation technology based on the large model can generate new synthetic samples by learning from existing sample data, thereby expanding the training set. This comprehensive approach not only improves the accuracy of sample annotation, but also enhances the overall performance of the image analysis model.
[0128] Example 2
[0129] This embodiment provides an electronic device, including: one or more processors and a memory, wherein the memory stores one or more programs, and the one or more programs include instructions for executing the large model-based power image sample data processing method as described in Example 1.
[0130] Example 3
[0131] This embodiment provides a computer-readable storage medium, including one or more programs for execution by one or more processors of an electronic device, wherein the one or more programs include instructions for executing the large model-based power image sample data processing method as described in Example 1.
[0132] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.
Claims
1. A method for processing power image sample data based on a large model, characterized in that: The steps include: Obtaining power images, using convolutional neural networks to extract low-level features and high-level features as image features, using the image features and pre-acquired power field knowledge base information as inputs of a pre-trained large model, generating label information, obtaining real samples, and realizing the annotation of power images; In the labeling process, Bayesian neural network is used to introduce probability distribution to maximize log-likelihood estimation and reduce labeling uncertainty. The goal is to train a generative adversarial network so that the discriminator cannot distinguish between generated samples and real samples. New power images are generated using the trained generative adversarial network, and generated samples are obtained through annotation. The mean absolute error and Fréchet distance are combined to evaluate the similarity and distribution consistency between real samples and generated samples.
2. The method for processing power image sample data based on a large model according to claim 1, characterized in that: The reduction of annotation uncertainty by maximizing log-likelihood estimation is implemented by the following formula: Among them, I is the power image of the model input data, is the predicted output of the model, θ is the model parameter, are the optimized model parameters, Given the input data I and parameter θ, the model output is The conditional probability of .
3. The method for processing power image sample data based on a large model according to claim 1, characterized in that: The loss function of the generative adversarial network is: min max L(G,D)=E Ireal～pdata(I) [log D(I real )]+E z～pz(z) [log(1-D(G(z)))] Among them, L(G,D) is the loss function of the generative adversarial network, G and D are the generator and discriminator respectively, and I real is the real image, p data (I) is the distribution of real data, p z (z) is the noise distribution of the generator input, z is the random noise input, G(z) is the new power image generated by the generator, D(I real ) is the discriminator for the real sample I real The output of D(G(z)) is the output of the discriminator for the generated sample G(z), E[·] is the expected value symbol, E Ireal～pdata(I) [log D(I real )] for real data sample I real The expected loss is calculated, E z～pz(z) [log(1-D(G(z)))] is the expected calculation of the loss of generating sample G(z).
4. The method for processing power image sample data based on a large model according to claim 1, characterized in that: The mean absolute error and Fréchet distance are calculated using the following formula: IS=1 / N∑|I i real -G(z) i |, FID=∥μ r -m g ∥ 2 +Tr(Σ r +S g -2(S r S g ) 1 / 2 ) Among them, MAE and FID are mean absolute error and Fréchet distance respectively, G(z) is the generated sample, I real is the real sample, N is the total number of pixels in the sample, μ r and μ g are the feature means of the real samples and the generated samples, Σ r and Σ g is the corresponding covariance matrix, and Tr() represents the trace operator of the matrix.
5. The method for processing power image sample data based on a large model according to claim 1, characterized in that: The image features are obtained using the following formula: F=CNN(I) Among them, F is the feature vector of image features, CNN is the convolutional neural network, and I is the power image.
6. The method for processing power image sample data based on a large model according to claim 1, characterized in that: The tag information is obtained using the following formula: L=NLP(F,K) Among them, L is the label information, NLP is the large model, F is the feature vector of image features, and K is the knowledge base information in the power field.
7. The method for processing power image sample data based on a large model according to claim 1, characterized in that: The label information includes defects, equipment abnormalities, equipment overheating, line breaks, insulation aging and normal operation.
8. The method for processing power image sample data based on a large model according to claim 1, characterized in that: The large model is a BERT model or a GPT model.
9. An electronic device, characterized in that: include: One or more processors and a memory, wherein the memory stores one or more programs, and the one or more programs include instructions for executing the large model-based power image sample data processing method as described in any one of claims 1-8.
10. A computer-readable storage medium, characterized in that: It comprises one or more programs for execution by one or more processors of an electronic device, wherein the one or more programs comprise instructions for executing the method for processing power image sample data based on a large model as claimed in any one of claims 1 to 8.