Noise image classification method, system and device based on federal multi-label graph neural network of generative data enhancement and medium

Abstract

The invention discloses a noise image classification method, system and device based on a federated multi-label graph neural network of generative data enhancement, and a medium, and belongs to the technical field of power system operation and maintenance, and the method comprises the steps: constructing a federated learning framework, defining a category name, extracting image features from a private data set, and constructing a local neural network model; performing comparative learning on the local neural network model and the global neural network model, screening high-confidence samples, generating an enhanced sample subset through a WGAN generator, and updating parameters of the local neural network model; calculating an unbalance coefficient, performing weighted aggregation to generate global model parameters, and deploying the global model parameters to a client; according to the federal multi-label graph neural network noise image classification method based on generative data enhancement, the problem that the performance of existing federal learning in a multi-label electric power image classification scene with class imbalance and label noise coexistence is severely reduced is effectively solved.

Description

Technical Field

[0001] This invention relates to the field of power system operation and maintenance technology, specifically to a method, system, device, and medium for classifying noisy images based on a federated multi-label graph neural network with generative data augmentation. Background Technology

[0002] Multi-label classification of power system operation scenarios is a key research focus in developing intelligent monitoring and diagnostic capabilities for modern smart grids. However, this task faces privacy constraints because operational data such as substation images and sensor records often contain sensitive infrastructure information. Federated learning (FL) offers an effective solution through a decentralized model training framework that supports collaborative knowledge extraction from geographically distributed data repositories. This approach adheres to strict data privacy standards while maintaining classification performance.

[0003] While Federated Learning (FL) has broad applicability across various domains, its deployment in power scene recognition faces unique challenges. Regional variations in environmental conditions and electrical equipment configurations result in heterogeneous image datasets, and multi-labeled scenes often exhibit class imbalance. Some classes maintain sufficient training samples, while others suffer from severe data scarcity, leading to degraded model performance on underrepresented classes. When these underrepresented classes are further contaminated by label noise, label noise from equipment degradation (e.g., sensor failure), environmental artifacts (e.g., weather interference), and inconsistent human annotations further degrades learning efficiency in distributed training paradigms. Commonly used FL methods primarily rely on perfectly labeled client datasets. This results in a lack of dedicated frameworks for noise-resistant optimization in class-imbalanced multi-label learning environments. Standard FL methods exhibit performance degradation during model aggregation due to the propagation of label noise across the system. In multi-label tasks, inherent label correlation further amplifies noise-induced errors, a problem that persists even in advanced frameworks such as FedProx. Furthermore, class imbalance amplifies label noise through distribution distortion. The combined effects of label noise and class imbalance impair the model's ability to learn representations of the minority class, limiting its performance on these data regions. Summary of the Invention

[0004] In view of the above-mentioned problems, the present invention is proposed.

[0005] Therefore, the technical problem solved by this invention is that commonly used federated learning methods lack a dedicated framework for noise-resistant optimization in class-imbalanced multi-label learning environments. Due to the propagation of label noise across the system, standard federated learning methods experience performance degradation during model aggregation. In multi-label tasks, inherent label correlation amplifies the error caused by noise, and this limitation exists even in high-level frameworks. Class imbalance amplifies label noise through distribution distortion, and this dual effect reduces the representation learning of minority classes, limiting the model's ability to represent these critical data regions.

[0006] To address the aforementioned technical problems, this invention provides the following technical solution: a noisy image classification method based on a federated multi-label graph neural network with generative data augmentation, comprising the following steps: A federated learning framework is constructed, in which each client maintains its own private dataset locally. Define category names and extract image features from a private dataset, then construct a local neural network model based on the image features and category names; The local neural network model is compared and learned from the global neural network model distributed by the server, and high-confidence samples are selected to generate a clean sample subset. An enhanced subset of samples is generated from a clean subset of samples using the WGAN generator, and the local model parameters of the local neural network model are updated based on the enhanced subset of samples. Calculate the imbalance coefficient based on the number of samples in each category name, and then perform weighted aggregation based on the imbalance coefficient and local model parameters to generate global model parameters. The global model parameters are deployed to the client. When a new noisy image is input, multi-label classification is performed on the new noisy image to generate classification results.

[0007] As a preferred embodiment of the noisy image classification method based on a federated multi-label graph neural network with generative data augmentation described in this invention, the steps of constructing a federated learning framework, wherein each client in the federated learning framework locally maintains a private dataset include: Deploy a federated learning system that includes a set number of clients, each of which stores a private dataset in its local private environment; The server initializes a global neural network model parameter and distributes the global neural network model parameter to all clients.

[0008] As a preferred embodiment of the noisy image classification method based on a federated multi-label graph neural network with generative data augmentation described in this invention, the method is characterized by the following steps: defining category names and extracting image features from a private dataset, and constructing a local neural network model based on the image features and category names. Define category names, which correspond to the multi-label semantic concepts to be identified. Convert each category name into a natural language text prompt and generate a set of label embeddings. Using the label embedding vectors in the label embedding set as graph nodes, the edge weights are determined based on the cosine similarity between any two graph nodes to construct a label-related graph. The label correlation graph is input into a graph attention autoencoder to generate embedded features; Image features are extracted from a private dataset by combining an attention mechanism with a neural network. Image features and embedded features are combined using a multilayer perceptron and input into a local neural network model.

[0009] As a preferred embodiment of the noisy image classification method based on a federated multi-label graph neural network with generative data augmentation described in this invention, the step of comparing and learning the local neural network model with the global neural network model distributed by the server, and selecting high-confidence samples to generate a clean sample subset includes: For each sample in the local private dataset, forward inference is performed using both a local neural network model and a global neural network model distributed by the server to generate a local prediction distribution and a global prediction distribution. Calculate the Wasserstein distance between the local and global prediction distributions; A clean subset of samples is generated by filtering high-confidence samples based on Wasserstein distance.

[0010] The beneficial effects of this preferred technical solution are as follows: by introducing label correlation graphs and graph attention mechanisms, semantic dependencies between multiple labels are established, alleviating semantic ambiguity caused by label noise; at the same time, by combining the contrastive learning framework of local and global models to screen high-confidence samples, noise propagation is suppressed, and reliable supervision signals are provided for subsequent generative data augmentation, thereby improving the accuracy of the model in identifying minority classes and the overall classification performance.

[0011] As a preferred embodiment of the noisy image classification method based on a federated multi-label graph neural network with generative data augmentation described in this invention, the steps of generating an augmented sample subset from a clean sample subset using a WGAN generator, and updating the local model parameters of the local neural network model based on the augmented sample subset include: The Wasserstein generative adversarial network is trained using a gradient-penalized Wasserstein loss function based on a clean subset of samples. Random noise is sampled and an enhanced subset of samples is generated using a Wasserstein generative adversarial network; The augmented sample subset and the clean sample subset are combined into an augmented training set. The local backbone model is trained using the augmented training set to update the local neural network model's local model parameters, thus obtaining the updated local model parameters.

[0012] As a preferred embodiment of the noisy image classification method based on a federated multi-label graph neural network with generative data augmentation described in this invention, the steps of calculating the imbalance coefficient based on the category name and sample quantity, and generating global model parameters by weighted aggregation based on the imbalance coefficient and local model parameters include: Count the number of samples for each category name in the clean sample subset for each client, and calculate the imbalance coefficient based on the sample count; Calculate the global model parameters based on the updated local model parameters and the imbalance coefficient.

[0013] The beneficial effects of this preferred technical solution are as follows: it uses high-confidence clean samples to drive the Wasserstein generative adversarial network to generate minority class synthetic images, thereby alleviating class imbalance; and it introduces a class imbalance coefficient in the aggregation stage to weight and fuse the parameters of the neural network model and the Wasserstein generative adversarial network, thereby improving noise resistance and data augmentation capabilities.

[0014] As a preferred embodiment of the noisy image classification method based on a federated multi-label graph neural network with generative data augmentation described in this invention, the steps of deploying global model parameters to the client and performing multi-label classification to generate classification results on the new noisy image when a new noisy image is input include: Deploy global model parameters to any client participating in federated learning; The new noisy image is input into the global neural network model for forward inference, and the corresponding classification result is output.

[0015] This invention provides a noisy image classification system based on a federated multi-label graph neural network with generative data augmentation.

[0016] To address the aforementioned technical problems, the present invention further provides the following technical solution: a noisy image classification system based on a federated multi-label graph neural network with generative data augmentation, comprising: Framework building module: Constructs the federated learning framework, in which each client maintains its own private dataset locally; Model building module: Defines category names and extracts image features from a private dataset, and builds a local neural network model based on the image features and category names; The contrastive learning module compares and learns with the global neural network model distributed by the server, and selects high-confidence samples to generate a clean sample subset. Parameter update module: Generates an enhanced sample subset based on the clean sample subset using the generator of the Wasserstein generative adversarial network, and updates the local model parameters of the local neural network model based on the enhanced sample subset; Data calculation module: Calculates the imbalance coefficient based on the category name and sample quantity, and generates global model parameters by weighted aggregation based on the imbalance coefficient and local model parameters; Image classification module: Deploys global model parameters to the client. When a new noisy image is input, it performs multi-label classification on the new noisy image to generate classification results.

[0017] The present invention provides a computer device, including a memory and a processor, wherein the memory stores a computer program, characterized in that the processor executes the computer program to implement the steps of a noisy image classification method based on a federated multi-label graph neural network with generative data augmentation.

[0018] The present invention provides a computer-readable storage medium having a computer program stored thereon, characterized in that, when the computer program is executed by a processor, it implements the steps of a noisy image classification method based on a federated multi-label graph neural network with generative data augmentation.

[0019] The beneficial effects of this invention are as follows: This invention constructs a federated multi-label graph neural network-based noisy image classification method based on generative data augmentation, effectively solving the problem of severe performance degradation in existing federated learning scenarios involving class imbalance and label noise in multi-label power image classification. Specifically, it includes introducing label correlation graphs and graph attention mechanisms, and combining a contrastive learning framework of local and global models to screen high-confidence samples to suppress noise propagation; simultaneously, it utilizes high-confidence clean samples to drive a Wasserstein generative adversarial network to generate minority class synthetic images, alleviating class imbalance; in the aggregation stage, it introduces a class imbalance coefficient to weight and fuse the parameters of the neural network model and the Wasserstein generative adversarial network, improving noise resistance and data augmentation capabilities; it solves the vicious cycle of label noise and class imbalance mutually amplifying each other, improves the model's ability to learn representations of the minority class, and is suitable for privacy-sensitive, heterogeneous, and noisy smart grid image recognition scenarios. Attached Figure Description

[0020] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1The above is a flowchart of a noisy image classification method based on a federated multi-label graph neural network with generative data augmentation, provided as an embodiment of the present invention.

[0022] Figure 2 This is a system architecture diagram of a noisy image classification method based on a federated multi-label graph neural network with generative data augmentation, provided as an embodiment of the present invention. Detailed Implementation

[0023] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.

[0024] Example 1, referring to Figure 1 This is the first embodiment of the present invention, which provides a noisy image classification method based on a federated multi-label graph neural network with generative data augmentation, comprising: S100: Construct a federated learning framework, in which each client in the federated learning framework maintains a private dataset locally; S200: Define category names and extract image features from a private dataset, and build a local neural network model based on the image features and category names; S300: Compare and learn from the local neural network model with the global neural network model distributed by the server, and select high-confidence samples to generate a clean sample subset; S400: Generate an enhanced sample subset from the clean sample subset using the WGAN generator, and update the local model parameters of the local neural network model based on the enhanced sample subset; S500: Calculate the imbalance coefficient based on the sample size of the category name, and generate global model parameters by weighted aggregation based on the imbalance coefficient and local model parameters; S600: Deploys global model parameters to the client. When a new noisy image is input, it performs multi-label classification on the new noisy image to generate classification results.

[0025] It should be noted that commonly used federated learning methods mainly rely on perfectly labeled client datasets. This leads to a lack of dedicated frameworks for noise-resistant optimization in imbalanced multi-label learning environments. Due to the propagation of label noise across the system, standard federated learning methods exhibit performance degradation during model aggregation. In multi-label tasks, inherent label correlation further amplifies noise-induced errors, a problem that persists even in advanced frameworks such as FedProx. Furthermore, class imbalance amplifies label noise through distribution distortion. The combined effect of label noise and class imbalance impairs the model's ability to learn representations of the minority class, limiting its performance on these data regions. Therefore, designing a neural network with label correlation modeling and combining dual-model contrastive learning for noise cleanup and class rebalancing is crucial.

[0026] Therefore, to address the aforementioned problem of label noise and class imbalance amplifying each other's errors, a noisy image classification method based on a federated multi-label graph neural network with generative data augmentation is constructed through steps S100-S600. This method introduces a label correlation graph and graph attention mechanism, combining a contrastive learning framework of local and global models to filter high-confidence samples to suppress noise propagation. Simultaneously, high-confidence clean samples are used to drive a Wasserstein generative adversarial network to generate synthetic minority class images, alleviating class imbalance. In the aggregation stage, a class imbalance coefficient is introduced to weight and fuse the parameters of the neural network model and the Wasserstein generative adversarial network, improving noise resistance and data augmentation capabilities. This solves the vicious cycle of label noise and class imbalance amplifying each other, and enhances the model's ability to learn representations of the minority class.

[0027] Example 2, refer to Figure 1 and Figure 2 This is the second embodiment of the present invention, which provides a noisy image classification method based on a federated multi-label graph neural network with generative data augmentation.

[0028] In this embodiment of the application, step S100 involves constructing a federated learning framework, where each client in the federated learning framework locally maintains its private dataset, including the following steps A1~A2: A1: Deploy a federated learning system with a specified number of clients, each client storing a private dataset in a local private environment; In this embodiment, the number is set to K, and a federated learning system with K clients is deployed in a smart grid scenario. Each client corresponds to a substation or regional monitoring node; each client independently stores the collected noise power image dataset in its local private environment. ,in Images of substations affected by rain, fog, ice, snow, and equipment aging. The corresponding multi-dimensional binary label vector represents whether the image contains predefined semantic categories such as insulator damage, transformer defects, and cluttered backgrounds. 1 indicates presence, and 0 indicates absence. The dimensions are determined based on the number of defined category names; A2: The server initializes a global neural network model parameter and distributes the global neural network model parameter to all clients.

[0029] In this embodiment, the central server initializes a global neural network model parameter in the initial training stage. The global neural network model structure is consistent with the local neural network model of the client and includes a convolutional attention layer and a multilayer perceptron classification head. The server sends the initialized global model parameter to each client through a secure communication channel. After receiving it, each client uses the initialized global model parameter as the starting point for local model training, which is used for subsequent comparative learning and parameter synchronization, thereby starting the federated training process.

[0030] In this embodiment of the application, step S200, which defines the category name and extracts image features from the private dataset, and constructs a local neural network model based on the image features and the category name, includes the following steps B1 to B5: B1: Define category names, which correspond to the multi-label semantic concepts to be identified. Convert each category name into a natural language text prompt and generate a tag embedding set. In this embodiment, C multi-label semantic category names for power scenarios are predefined, including equipment category and environmental interference category. The equipment category includes insulator damage and transformer, and the environmental interference category includes rain and fog interference and background clutter. For each category name, the category name is converted into a more complete and descriptive natural language sentence form text prompt, and processed by CLIP's frozen text encoder to generate the corresponding label embedding vector, thereby forming a label embedding set.

[0031] B2: Using the label embedding vectors in the label embedding set as graph nodes, determine the edge weights based on the cosine similarity between any two graph nodes, and construct a label-related graph. In this embodiment, the label embedding vector in the label embedding set is regarded as a node in the noisy power image. The cosine similarity between any two nodes is calculated, and the edge weight is calculated by the normalization function, thereby constructing a fully connected label correlation graph with C nodes for explicit modeling of semantic dependencies between categories. The formula for calculating edge weights is: in, The cosine similarity function is used. For the standardized function, and For any two nodes; B3: Input the label correlation graph into the graph attention autoencoder to generate embedded features; In this embodiment, the label correlation graph is input into the graph attention autoencoder, neighborhood information is aggregated through a learnable attention mechanism, the affinity between input vectors based on the neural network is calculated, and then a normalized attention coefficient is generated based on the affinity to quantify the relationship strength. Finally, the embedding features of the fused label semantic correlation are calculated based on the attention coefficient. In this process, the complex dependencies between labels in the label correlation graph are automatically learned. Even if some labels are noisy, stable semantic signals can be obtained through graph propagation. The formula for calculating the affinity between input vectors based on a neural network is: ; in, For affinity, and For any two nodes, embed the label vectors. For a learnable weight matrix, The adjacency matrix of the label-related graph. The cosine similarity function is used. It is a single-layer neural network; The formula for quantifying relationship strength by generating a normalized attention coefficient based on affinity is as follows: ; in, Attention coefficient For affinity, It is a natural exponential function; The original affinity scores are transformed into attention weights with clear probabilistic meaning through normalization; The formula for calculating the embedding features of semantic relevance of fused labels based on the attention coefficient is as follows: ; in, For embedding features, Embed the label vector for neighbor node j. For learnable weights, Attention coefficient; Calculating the embedded features based on the label embedding vector and attention coefficients to extract enhanced features suitable for specific tasks and containing rich structured knowledge is one of the key steps for effective multi-label recognition and noise resistance. B4: Extracting image features from a private dataset using an attention mechanism combined with a neural network; In this embodiment, the neural network employs a Backbone network. For each noisy power image in the client's local private dataset, a Backbone network containing convolutional layers and an attention mechanism is used to extract features and output image features. The expression for the output image features is: ; in, For image features, For attention mechanism functions, For neural network functions, A noisy power image; By extracting features from the output image, the original image is transformed into high-dimensional features suitable for classification, providing a high-quality visual foundation. The introduced attention mechanism enables the model to focus on key areas in the image, reducing the interference of background clutter and noise, which is particularly important for complex power field environments. In one alternative implementation, the neural network can also employ a Transformer model, which divides the input noisy power image into fixed-size image blocks, performs linear embedding, and then concatenates them for position encoding. The encoder of the multi-layer Transformer model is input to the Transformer model's self-attention mechanism, which can model the long-distance dependency between any two regions in the image. In complex power scenarios, the Transformer model can simultaneously capture local details and global semantic context, fusing multi-scale features into image features and suppressing interference caused by cluttered backgrounds.

[0032] In another alternative implementation, the neural network can also employ a multi-scale feature pyramid network. This network constructs multi-scale image features, using a convolutional neural network as the backbone. It extracts feature maps of different resolutions at multiple convolutional stages and fuses high-level semantic information and low-level detail information through top-down paths and lateral connections, generating a set of feature maps with the same semantic strength but different spatial resolutions. In power images, large equipment requires low-resolution features to capture the overall structure, while small defects require high-resolution features for precise localization. The multi-scale feature pyramid network can provide both types of information simultaneously and fuse multi-scale features into image features.

[0033] B5: Based on image features and embedded features, a multilayer perceptron is used to combine them and input them into a local neural network model.

[0034] In this embodiment, image features and embedded features are concatenated and input into a multilayer perceptron to form the classification output layer of a local neural network model. Steps B1-B3 are used as the label semantic modeling module of the local neural network model, and step B4 is used as the image feature extraction module to complete the construction of the local neural network model. This local neural network model can be used for forward inference and training of multi-label noisy images.

[0035] In this embodiment of the application, step S300, which compares and learns the local neural network model with the global neural network model issued by the server, and selects high-confidence samples to generate a clean sample subset, includes the following steps C1~C3: C1: For each sample in the local private dataset, forward inference is performed using both the local neural network model and the global neural network model distributed by the server to generate the local prediction distribution and the global prediction distribution. In this embodiment of the application, the client loads the local neural network model trained locally and the global neural network model sent by the server in the previous round of communication. For each noisy power image sample in the local private dataset, the client inputs the local neural network model and the global neural network model respectively to perform forward inference, and obtains the local prediction distribution and the global prediction distribution with the same dimension and the same number of category names. C2: Calculate the Wasserstein distance between the local and global prediction distributions; In this embodiment of the application, the formula for calculating the Wasserstein distance between the local prediction distribution and the global prediction distribution is: ; in, For Wasserstein distance, For local predicted distribution, For global prediction distribution, The Wasserstein distance function; It should be noted that the smaller the calculated Wasserstein distance, the more consistent the semantic judgments of the local neural network model and the global neural network model are on the same image, and the more likely the sample is to be a clean sample. C3: Use Wasserstein distance to select high-confidence samples to generate a clean sample subset.

[0036] In this embodiment of the application, a distance threshold is set, and the set threshold is compared with the Wasserstein distance calculated in step C2. If the Wasserstein distance is less than the set distance threshold, the sample is classified as a clean sample; if the Wasserstein distance is greater than or equal to the set distance threshold, the sample is discarded, and the clean samples are integrated into a clean sample subset.

[0037] In one alternative implementation, high-confidence samples can also be screened using binary consistency screening of the predicted intersection. For each sample, multidimensional logits are output by local and global models respectively. A uniform threshold is applied to generate a binary prediction vector. The ratio of the intersection to the union of the two binary vectors is calculated. If the sample is clean, the two models should predict a highly overlapping set of labels, and the ratio of the intersection to the union of the two binary vectors should be close to 1. If there is label noise, the prediction results are often significantly different, and the ratio of the intersection to the union of the two binary vectors is low. Then, a ratio threshold is set, and high-confidence samples are screened based on the ratio threshold.

[0038] In another alternative implementation, high-confidence samples can also be selected using KL divergence. For each noisy power image sample in the local private dataset, the client performs forward inference using a local neural network model and a global neural network model issued by the server to obtain two predicted probability distributions. The KL divergence between the two predicted probability distributions is calculated as a measure of inconsistency. KL divergence reflects the information loss of one distribution relative to the other. The smaller the value, the closer the predictions of the two models are. A KL divergence threshold is then set, and high-confidence samples are selected based on the ratio threshold.

[0039] In this embodiment of the application, step S400, which generates an enhanced sample subset based on a clean sample subset using a WGAN generator and updates the local model parameters of the local neural network model based on the enhanced sample subset, includes the following steps D1~D3: D1: Train the Wasserstein generative adversarial network using the gradient-penalized Wasserstein loss function on a clean subset of samples; In this embodiment, a clean sample subset is used as the real data distribution to construct a Wasserstein generative adversarial network. The generator in the Wasserstein network receives a random noise vector with category name labels as input. The loss function for constructing the Wasserstein network adopts a gradient-penalized form, specifically: ; in, For loss function, For the expected value of the real data item, For the true data distribution, The output score of the discriminator for the real sample x. To generate the expected data items, Let z be the distribution of the input noise z of the generator. The generator generates a distribution for a specified category based on the category label c. The generator takes noise z and class label c as input to generate a synthetic image. The output score of the discriminator for this synthesized image. This is a gradient penalty term; At the same time, all weight matrices of the discriminator are subjected to spectral normalization. ; in, The weight matrix after spectral normalization. This is the weight matrix. The singular values ​​of the weight matrix are denoted by σ(W). The spectral norm of the weight matrix (i.e., the largest singular value σ(W)) is constrained to 1 by spectral normalization. By forcing the spectral norm of the weight matrix to be 1, the Lipschitz constant of the entire discriminator network can be controlled within a small range, thereby rigidly ensuring that the discriminator satisfies the Lipschitz continuity condition. When training the Wasserstein generative adversarial network, an alternating optimization strategy is adopted. In each training round, the generator is fixed first, and the discriminator is updated by gradient ascent to maximize the loss function. Then, the discriminator is fixed and the generator is updated by gradient descent to minimize the discrimination score of the generated samples, thereby improving the semantic realism and category consistency of the generated images. The training continues until the confidence of the generated samples on the discriminator tends to stabilize and the visual quality of the synthesized images meets the requirements, thereby determining the parameters of the Wasserstein generative adversarial network. D2: Sample random noise and generate an enhanced subset of samples using a Wasserstein generative adversarial network; In this embodiment, after the Wasserstein generative adversarial network training chain is closed, the generator is fixed, and for each category name with insufficient sample quantity, random noise vectors following a standard normal distribution are sampled multiple times and combined with the category name to generate synthetic images. The generated images are scored and filtered by the discriminator, and only image samples with discriminator output scores higher than a set score threshold are retained to form an enhanced sample subset. D3: Combine the augmented sample subset and the clean sample subset into an augmented training set. Use the augmented training set to train the local neural network model and update the local model parameters to obtain the updated local model parameters.

[0040] In this embodiment, the average number of samples for all category names in the local dataset is calculated. For each category name, if its current number of samples is less than the average number of samples, synthetic samples are added from the augmented sample subset until the current number of samples for that category reaches the average number of samples. The added synthetic samples are then combined with the clean sample subset to form an augmented training set. The image samples in the augmented training set are used as input, and the corresponding binary multi-label vectors are used as supervision labels to supervise the training of the local neural network model. The parameters of its convolutional layer, attention module, and classification head are updated through backpropagation. When the improvement of the multi-label evaluation index on the local validation set is less than 0.1% for 5 consecutive rounds, the local neural network model is determined to have converged, and training is stopped. At this time, the parameters of the local neural network model are used to update the local model parameters.

[0041] In one alternative implementation, updating the local model parameters can also be achieved through gradient clipping. After calculating the local total loss gradient through backpropagation, the gradient vector is first clipped using the L2 norm to prevent gradient explosion. The clipped gradient is then combined with the historical momentum using an exponentially weighted average to form the update direction, thereby updating the local model parameters.

[0042] In another alternative implementation, the local model parameters can also be updated through differential learning rate updates for parameter grouping. The local model parameters are divided into three groups: the first group consists of learnable weights of the graph attention network, the second group consists of contrastive learning projector parameters, and the third group consists of link prediction classifier parameters. Different learning rates are assigned to each group. During local training, the parameters of each group are updated independently according to their respective learning rates, thereby updating the local model parameters.

[0043] In this embodiment of the application, step S500, which calculates the imbalance coefficient based on the number of samples according to the category name, and generates global model parameters by weighted aggregation based on the imbalance coefficient and local model parameters, includes the following steps E1~E2: E1: Count the number of samples for each category name in the clean sample subset for each client, and calculate the imbalance coefficient based on the number of samples; In this embodiment of the application, for each client, the number of samples for each category name in the clean sample subset is counted, and the formula for calculating the imbalance coefficient based on the sample number is as follows: ; in, The imbalance coefficient is C, where C is the total number of category names. This represents the number of samples for each category name in the clean sample subset. The number of samples corresponding to the category name with the most samples; It should be noted that the imbalance coefficient is used to quantify the degree of imbalance in the distribution of samples of each category in the local data of each client. The larger the value of the imbalance coefficient, the more severe the class imbalance of the client. In one alternative implementation, the imbalance coefficient can also be calculated by quantifying the class distribution equilibrium of Shannon entropy. This involves counting the number of samples in each class within a clean sample subset and calculating the class frequency distribution. Based on the frequency distribution, Shannon entropy is calculated. The imbalance coefficient is defined as the difference between the maximum value of Shannon entropy under ideal equilibrium conditions and the Shannon entropy.

[0044] In another alternative implementation, the imbalance coefficient can also be calculated based on semantic weighting of gradient direction consistency. After training locally, the client uploads the gradient vector of the loss function, and the server calculates the cosine similarity between each client's gradient and the group's average gradient as the imbalance coefficient.

[0045] E2: Calculate the global model parameters based on the updated local model parameters and the imbalance coefficient.

[0046] In this embodiment of the application, each client uploads the local parameters of the Wasserstein generative adversarial network determined in step D1, the updated local model parameters obtained in step D3, and the imbalance coefficient obtained in step E1 to the central server. The central server performs weighted aggregation on the local parameters of the Wasserstein generative adversarial network and the updated local model parameters to calculate the global Wasserstein generative adversarial network parameters and the global neural network model parameters, respectively. The formula for calculating the parameters of the global Wasserstein generative adversarial network is: ; in, For global Wasserstein generative adversarial network parameters, The local parameters of the Wasserstein generative adversarial network determined in step D1 are... The imbalance coefficient, Total number of clients The calculated global Wasserstein generative adversarial network parameters are used to generate an enhanced subset of samples in the next round. The formula for calculating the parameters of the global neural network model is: ; in, These are the parameters of the global neural network model. The updated local model parameters obtained in step D3 The imbalance coefficient, Total number of clients Index for the client; The global neural network model parameters and the global Wasserstein generative adversarial network parameters are output as global model parameters. It should be noted that using the imbalance coefficient as the basis for aggregation weights allows the server to assign higher weights to clients with more severe imbalances when aggregating the global model, thereby enhancing the representation of minority classes in the global model and alleviating the problem of majority classes dominating and minority classes being submerged. The imbalance coefficient should be calculated based on a clean subset of samples to avoid interference from label noise on the imbalance evaluation.

[0047] In this embodiment of the application, step S600, which deploys the global model parameters to the client, and when a new noisy image is input, performs multi-label classification on the new noisy image to generate classification results, includes the following steps F1~F2: F1: Deploy global model parameters to any client participating in federated learning; In this embodiment of the application, after completing the weighted aggregation in step D2, the central server distributes and deploys the generated global neural network model parameters from the global model parameters to any client participating in federated learning. These global neural network model parameters have integrated the noise resistance knowledge and class balance capabilities of each client and can independently perform multi-label classification tasks. F2: Input the new noisy image into the global neural network model for forward inference and output the corresponding classification result.

[0048] In this embodiment, a new noisy power image to be classified is input into a deployed global neural network model. The global neural network model extracts image features through a convolutional attention network, then fuses them with embedded features, passes them through a multilayer perceptron, and activates them with a sigmoid function to output a multi-label classification result with the same dimension as the number of category names. This indicates whether the image contains a semantic category with defined category names, thus achieving multi-label semantic recognition of noisy images. This method can complete the semantic parsing of noisy power images on any client without uploading any original image data and without relying on centralized computing.

[0049] This invention constructs a federated multi-label graph neural network-based noisy image classification method based on generative data augmentation. It effectively addresses the severe performance degradation of existing federated learning methods in multi-label power grid image classification scenarios where class imbalance and label noise coexist. Specifically, it introduces label correlation graphs and graph attention mechanisms, combining a contrastive learning framework of local and global models to filter high-confidence samples to suppress noise propagation. Simultaneously, it utilizes high-confidence clean samples to drive a Wasserstein generative adversarial network to generate minority class synthetic images, alleviating class imbalance. In the aggregation stage, a class imbalance coefficient is introduced to weight and fuse the parameters of the neural network model and the Wasserstein generative adversarial network, improving noise resistance and data augmentation capabilities. This method resolves the vicious cycle of label noise and class imbalance amplifying each other, enhances the model's ability to learn representations of the minority class, and is suitable for privacy-sensitive, heterogeneous, and noisy smart grid image recognition scenarios.

[0050] Example 3, referring to Figure 1 This is a third embodiment of the present invention, which provides a noisy image classification system based on a federated multi-label graph neural network with generative data augmentation, comprising: Framework building module: Constructs the federated learning framework, in which each client maintains its own private dataset locally; Model building module: Defines category names and extracts image features from a private dataset, and builds a local neural network model based on the image features and category names; The contrastive learning module compares and learns with the global neural network model distributed by the server, and selects high-confidence samples to generate a clean sample subset. Parameter update module: Generates an enhanced sample subset based on the clean sample subset using the WGAN generator, and updates the local model parameters of the local neural network model based on the enhanced sample subset; Data calculation module: Calculates the imbalance coefficient based on the category name and sample quantity, and generates global model parameters by weighted aggregation based on the imbalance coefficient and local model parameters; Image classification module: Deploys global model parameters to the client. When a new noisy image is input, it performs multi-label classification on the new noisy image to generate classification results.

[0051] Example 4, the fourth embodiment of the present invention, differs from the previous three embodiments in that: if the function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0052] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-including system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0053] More specific examples of computer-readable media (a non-exhaustive list) include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0054] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0055] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A noisy image classification method based on a federated multi-label graph neural network with generative data augmentation, characterized in that, include, A federated learning framework is constructed, in which each client maintains its own private dataset locally. Define category names and extract image features from a private dataset; construct a local neural network model based on the image features and category names. The local neural network model is compared and learned from the global neural network model distributed by the server, and high-confidence samples are selected to generate a clean sample subset. An enhanced subset of samples is generated from a clean subset of samples using the WGAN generator, and the local model parameters of the local neural network model are updated based on the enhanced subset of samples. Calculate the imbalance coefficient based on the number of samples in each category name, and then perform weighted aggregation based on the imbalance coefficient and local model parameters to generate global model parameters. The global model parameters are deployed to the client. When a new noisy image is input, multi-label classification is performed on the new noisy image to generate classification results.

2. The noisy image classification method based on a federated multi-label graph neural network with generative data augmentation as described in claim 1, characterized in that, The steps involved in building a federated learning framework, where each client maintains its own private dataset locally, include: Deploy a federated learning system that includes a set number of clients, each of which stores a private dataset in its local private environment; The server initializes a global neural network model parameter and distributes the global neural network model parameter to all clients.

3. The noisy image classification method based on a federated multi-label graph neural network with generative data augmentation as described in claim 2, characterized in that, The steps involved in defining category names, extracting image features from a private dataset, and constructing a local neural network model based on the image features and category names include: Define category names, which correspond to the multi-label semantic concepts to be identified. Convert each category name into a natural language text prompt and generate a set of label embeddings. Using the label embedding vectors in the label embedding set as graph nodes, the edge weights are determined based on the cosine similarity between any two graph nodes to construct a label-related graph. The label correlation graph is input into a graph attention autoencoder to generate embedded features; Image features are extracted from a private dataset by combining an attention mechanism with a neural network. Image features and embedded features are combined using a multilayer perceptron and input into a local neural network model.

4. The noisy image classification method based on a federated multi-label graph neural network with generative data augmentation as described in claim 3, characterized in that, The steps involved in comparing and learning from the local neural network model with the global neural network model distributed by the server, and then selecting high-confidence samples to generate a clean sample subset, include: For each sample in the local private dataset, forward inference is performed using both a local neural network model and a global neural network model distributed by the server to generate a local prediction distribution and a global prediction distribution. Calculate the Wasserstein distance between the local and global prediction distributions; A clean subset of samples is generated by filtering high-confidence samples based on Wasserstein distance.

5. The noisy image classification method based on a federated multi-label graph neural network with generative data augmentation as described in claim 4, characterized in that, The steps of generating an augmented sample subset from a clean sample subset using the WGAN generator, and updating the local model parameters of the local neural network model based on the augmented sample subset include: The Wasserstein generative adversarial network is trained using a gradient-penalized Wasserstein loss function based on a clean subset of samples. Random noise is sampled and an enhanced subset of samples is generated using a Wasserstein generative adversarial network; The augmented sample subset and the clean sample subset are combined into an augmented training set. The local backbone model is trained using the augmented training set to update the local neural network model's local model parameters, thus obtaining the updated local model parameters.

6. The noisy image classification method based on a federated multi-label graph neural network with generative data augmentation as described in claim 5, characterized in that, The steps involved in calculating the imbalance coefficient based on the category name and sample size, and then generating global model parameters by weighted aggregation based on the imbalance coefficient and local model parameters, are as follows: Count the number of samples for each category name in the clean sample subset for each client, and calculate the imbalance coefficient based on the sample count; Calculate the global model parameters based on the updated local model parameters and the imbalance coefficient.

7. The noisy image classification method based on a federated multi-label graph neural network with generative data augmentation as described in claim 6, characterized in that, Deploying global model parameters to the client and performing multi-label classification to generate classification results on a new noisy image as input includes the following steps: Deploy global model parameters to any client participating in federated learning; The new noisy image is input into the global neural network model for forward inference, and the corresponding classification result is output.

8. A noisy image classification system based on a federated multi-label graph neural network with generative data augmentation, employing the noisy image classification method based on a federated multi-label graph neural network with generative data augmentation as described in any one of claims 1 to 7, characterized in that, include: Framework building module: Constructs the federated learning framework, in which each client maintains its own private dataset locally; Model building module: Defines category names and extracts image features from a private dataset, and builds a local neural network model based on the image features and category names; The contrastive learning module compares and learns with the global neural network model distributed by the server, and selects high-confidence samples to generate a clean sample subset. Parameter update module: Generates an enhanced sample subset based on the clean sample subset using the generator of the Wasserstein generative adversarial network, and updates the local model parameters of the local neural network model based on the enhanced sample subset; Data calculation module: Calculates the imbalance coefficient based on the category name and sample quantity, and generates global model parameters by weighted aggregation based on the imbalance coefficient and local model parameters; Image classification module: Deploys global model parameters to the client. When a new noisy image is input, it performs multi-label classification on the new noisy image to generate classification results.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the noisy image classification method based on a federated multi-label graph neural network according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the noisy image classification method based on a federated multi-label graph neural network according to any one of claims 1 to 7.

Images