Method, system and equipment for constructing multi-label image recognition model based on federated learning and medium

Abstract

The invention discloses a method, a system and equipment for constructing a multi-label image recognition model based on federated learning, and a medium, and the method comprises the steps: carrying out the label embedding processing of a multi-label image data set, and obtaining a mask label embedding vector of each client; performing feature extraction on the tag association graph through a graph attention auto-encoder to obtain tag encoding features; fusing the label coding features and the image features, and inputting the fused features into a classifier for training to obtain a local model; each client calculates an unbalance coefficient of each category based on the number of samples of each category in the multi-label image data set; and the server divides the local model parameters into a plurality of model groups through a clustering algorithm, calculates an average value of unbalance coefficients of clients in each model group, and performs weighted aggregation on each group of aggregation models based on the average value to obtain a global model. The problems that in existing multi-label image recognition model construction, label association is not fully mined, and label embedding lacks dynamic adaptation are solved.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and in particular to a method, system, device and medium for constructing a multi-label image recognition model based on federated learning. Background Technology

[0002] The demand for multi-label image recognition in fields such as security monitoring and medical image analysis continues to grow. However, image data in these scenarios is often stored in different terminal devices, and the requirements for data privacy protection are becoming increasingly stringent. Therefore, distributed model training based on federated learning has gradually become a research focus for this type of task. However, multi-label image recognition itself still faces problems such as complex relationships between labels and unbalanced distribution of data categories. How to improve the performance of multi-label recognition while ensuring data privacy has become a practical issue that needs to be addressed.

[0003] However, existing technologies often fail to fully explore the relationships between labels when building multi-label image recognition models based on federated learning. The label embedding process also lacks dynamic adjustment adaptability. In the face of the imbalanced class distribution of data from different clients in distributed scenarios, there are no targeted coefficients to match this difference. At the same time, the fusion of label features and image features is not tight enough during the local model training stage, and the similarity of local model parameters is not combined for group optimization during the subsequent global model aggregation. This results in the trained model either underutilizing label association features or failing to adapt to the imbalanced data from different clients, ultimately affecting the performance of multi-label recognition and the generalization ability of the model. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, this invention provides a method, system, device, and medium for constructing a multi-label image recognition model based on federated learning to solve the problems in the construction of existing multi-label image recognition models based on federated learning, such as insufficient exploration of label associations, lack of dynamic adaptation of label embedding, no measures to match data imbalance, loose feature fusion, and unoptimized global aggregation, resulting in poor model adaptation and insufficient recognition and generalization capabilities.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution: In a first aspect, the present invention provides a method for constructing a multi-label image recognition model based on federated learning, comprising: acquiring multi-label image datasets from each client in a federated learning system, and performing label embedding processing on the multi-label image datasets to obtain masked label embedding vectors for each client; constructing a label association graph based on the masked label embedding vectors, and extracting features from the label association graph using a graph attention autoencoder to obtain label encoded features; each client extracting image features from the multi-label image dataset, fusing the label encoded features with the image features, and inputting the fused features into a classifier for training to obtain a local model; each client calculating the imbalance coefficients for each category based on the number of samples in each category of the multi-label image dataset, and uploading the model parameters and imbalance coefficients of the local model to a server; the server dividing the local model parameters into multiple model groups using a clustering algorithm, and aggregating the parameters within each model group to obtain multiple group aggregated models, calculating the average value of the imbalance coefficients of the clients within each model group, and performing weighted aggregation on each group aggregated model based on the average value to obtain a global model.

[0007] As a preferred embodiment of the construction method of the multi-label image recognition model based on federated learning described in this invention, the step of performing label embedding processing on the multi-label image dataset to obtain the mask label embedding vector of each client includes: encoding the labels in the multi-label image dataset to generate label embedding vectors, and initializing the state embedding vector based on the true state of the labels; during the training iteration, predicting the multi-label image dataset to obtain the prediction confidence of each label, and updating the state embedding vector according to the relationship between the prediction confidence and a preset threshold; performing element-wise multiplication of the label embedding vector and the updated state embedding vector to obtain the mask label embedding vector.

[0008] The beneficial effects of this preferred technical solution are as follows: Labels in a multi-label image dataset are encoded to generate label embedding vectors, and state embedding vectors are initialized based on the true state of the labels. During training iterations, the state embedding vectors are updated based on the relationship between the predicted confidence of each label and a preset threshold. Then, the label embedding vectors and the updated state embedding vectors are multiplied element-wise to obtain masked label embedding vectors. This process incorporates the prediction feedback information during training into the label embedding vectors, allowing the masked label embedding vectors to simultaneously carry both the original label features and the predicted state of the labels. This ensures that the presentation of label-related features aligns with the actual training process of the model, avoiding the situation where the original label embedding remains fixed during training and cannot adapt to the dynamic prediction. Subsequent feature processing based on these masked label embedding vectors can rely on label information that matches the training process, making the utilization of label features more consistent with the real-time state of model training.

[0009] As a preferred embodiment of the construction method for a multi-label image recognition model based on federated learning described in this invention, the step of extracting features from the label association graph using a graph attention autoencoder to obtain label-encoded features includes: calculating the similarity between each label node in the label association graph based on the dot product operation between the masked label embedding vectors, normalizing the similarity, and using the normalized similarity as the edge weight of the label association graph; calculating the attention coefficient of each label node based on the similarity, and weighting and aggregating the features of other label nodes connected to the current label node according to the attention coefficient to obtain the attention features of each label node; decoding and reconstructing the attention features using the decoder of the graph attention autoencoder, and training the encoder of the graph attention autoencoder based on the difference between the original features and the reconstructed features to obtain the label-encoded features.

[0010] The beneficial effects of this preferred technical solution are as follows: The similarity of each label node in the label association graph is calculated based on the dot product operation between the masked label embedding vectors and normalized as edge weights, allowing the edge weights to reflect the degree of association between label nodes. Then, the attention coefficient of each label node is calculated based on the similarity. The features of other connected label nodes are weighted and aggregated according to the attention coefficients to obtain attention features, ensuring that the features of each label node incorporate relevant information about the associated labels. The attention features are decoded and reconstructed using the decoder of a graph attention autoencoder. The difference between the original features and the reconstructed features is used to train the encoder to obtain label encoding features. This allows the label encoding features to simultaneously carry the characteristics of the label itself and the association information between labels, matching the actual needs of label feature extraction.

[0011] As a preferred embodiment of the method for constructing a multi-label image recognition model based on federated learning as described in this invention, the step of fusing the label-encoded features with image features and then inputting them into a classifier for training to obtain a local model includes: flattening the label-encoded features into a one-dimensional feature vector, and concatenating the one-dimensional feature vector with image features extracted from the multi-label image dataset to obtain fused features; inputting the fused features into a feature transformation network in the classifier for feature transformation, and outputting the prediction results of each label through the fully connected classification layer of the classifier; constructing a loss function, which includes a classification loss term, a reconstruction loss term, and a regularization loss term, and training the local models of each client based on the loss function.

[0012] The beneficial effects of this preferred technical solution are as follows: After flattening the label encoding features into a one-dimensional feature vector, it is concatenated with the image features to obtain the fused features. The fused features simultaneously carry label association information and image visual information, avoiding the information limitations caused by a single feature dimension. The fused features are input into the feature transformation network of the classifier for processing, and then output through the fully connected classification layer to produce the prediction results of each label. This enables the fused features to be transformed into targeted recognition prediction outputs. A loss function containing classification loss terms, reconstruction loss terms, and regularization loss terms is constructed. Using such a loss function to train the local models of each client can constrain the training process from multiple dimensions such as classification accuracy, feature reconstruction effect, and parameter regularity. This allows the training of the local model to take into account both label prediction accuracy and feature representation rationality, making the learning process of the local model fit the task essence of multi-label recognition, and ensuring that the output of the trained local model matches the actual recognition requirements.

[0013] As a preferred embodiment of the method for constructing a multi-label image recognition model based on federated learning as described in this invention, the loss function is specifically expressed as: ; In the formula, Loss is the total loss. For classifying losses, The reconstruction loss is the difference between the original features and the reconstructed features, and w represents the parameters of the local model. These are the parameters of the global model. The regularization coefficient is . It is the squared Euclidean norm of the difference between the local model parameters and the global model parameters.

[0014] The beneficial effects of this preferred technical solution are as follows: By setting the total loss to include both classification loss and reconstruction loss, the loss function simultaneously constrains the label prediction results and the differences between the original features and the reconstructed features. Furthermore, by adding the Euclidean norm squared term of the difference between the local model parameters and the global model parameters, combined with the regularization coefficient, the parameters of the local model maintain a reasonable correlation with the parameters of the global model during training. This loss function can constrain the training process of the local model from several dimensions, including classification performance, feature reconstruction effect, and the correlation between the parameters of the local model and the global model. This allows the training of the local model to simultaneously take into account the prediction requirements of multi-label recognition and the rationality of feature representation. The trained local model parameters not only fit the data characteristics of the client but also adapt to the parameter trends of the global model, thus matching the training requirements of multi-label image recognition models under the federated learning framework.

[0015] As a preferred embodiment of the construction method for the multi-label image recognition model based on federated learning described in this invention, the step of calculating the imbalance coefficient of each category based on the number of samples in each category of the multi-label image dataset includes: counting the total number of samples and the number of samples in each category of the multi-label image dataset, and calculating the imbalance coefficient, wherein the calculation of the imbalance coefficient is expressed as: ; In the formula, Let be the imbalance coefficient for category c. Where C is the total number of samples and C is the total number of categories. The number of samples in category c.

[0016] The beneficial effects of this preferred technical solution are as follows: It statistically analyzes the total number of samples and the number of samples in each category in a multi-label image dataset, clarifies the distribution of samples of different categories within the dataset, and then calculates the class imbalance coefficient based on the total number of samples, the total number of categories, and the number of samples in the corresponding category. This allows the imbalance coefficient to quantify the difference between the number of samples in each category and the overall sample distribution, ensuring that the degree of imbalance in each category has a corresponding numerical value. At the same time, it can closely match the actual sample distribution of the dataset, addressing the distribution differences of samples in different categories, and matching the actual situation of uneven class distribution in multi-label image datasets. The calculation result of the imbalance coefficient is directly related to the sample composition of the dataset, and the value can accurately correspond to the proportion of each category in the dataset.

[0017] As a preferred embodiment of the method for constructing a multi-label image recognition model based on federated learning as described in this invention, the step of dividing local model parameters into multiple model groups using a clustering algorithm and aggregating parameters within each model group to obtain multiple group aggregated models includes: extracting the parameters of the classifier in each local model as clustering features, and dividing each local model into multiple model groups based on the similarity between the parameters of the classifier using a clustering algorithm; calculating weight coefficients according to the similarity between the parameters of each local model within each model group, and performing a weighted summation of the local model parameters within each model group according to the weight coefficients to obtain the group aggregated model corresponding to each model group.

[0018] The beneficial effects of this preferred technical solution are as follows: The parameters of the classifiers in each local model are extracted as clustering features. Based on the similarity between these parameters, multiple model groups are divided using a clustering algorithm. This allows the division of model groups to be directly based on the correlation characteristics of the core parameters of the local models, ensuring that the parameters of the local models within a group have similar feature tendencies. Then, weight coefficients are calculated based on the similarity between the parameters of the local models within each model group. The local model parameters within the model group are then weighted and summed according to these weight coefficients to obtain the group aggregation model. This allows the parameter aggregation process to reflect the parameter differences of each local model within the model group, enabling the group aggregation model to integrate the training information of each local model within the model group. The aggregation result closely matches the actual parameter situation of the models within the model group, thus meeting the actual needs of global model aggregation in federated learning.

[0019] Secondly, the present invention provides a system for constructing a multi-label image recognition model based on federated learning, comprising: Data acquisition and label embedding module: used to acquire multi-label image datasets from each client in the federated learning system, and to perform label embedding processing on the multi-label image datasets to obtain the mask label embedding vectors of each client; Feature extraction module: used to construct a label association graph based on the mask label embedding vector, and to extract features from the label association graph through a graph attention autoencoder to obtain label encoding features; Local model training module: This module is used by each client to extract image features from a multi-label image dataset, fuse the label-encoded features with the image features, and then input the fused features into a classifier for training to obtain a local model. Parameter calculation and upload module: used by each client to calculate the imbalance coefficient of each category based on the number of samples of each category in the multi-label image dataset, and upload the model parameters and imbalance coefficients of the local model to the server; Global model aggregation module: The server uses a clustering algorithm to divide local model parameters into multiple model groups, aggregates parameters within each model group to obtain multiple aggregated models, calculates the average value of the imbalance coefficient of the client within each model group, and performs weighted aggregation on each aggregated model group based on the average value to obtain the global model.

[0020] Thirdly, the present invention provides an electronic device, comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, which, when executed by the processor, implement the steps of a method for constructing a multi-label image recognition model based on federated learning.

[0021] Fourthly, the present invention provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the method for constructing the multi-label image recognition model based on federated learning.

[0022] Compared with existing technologies, the beneficial effects of this invention are as follows: Based on the federated learning framework, it is specifically optimized for the actual needs of multi-label image recognition, improving the overall performance of the global model without leaking the data privacy of distributed terminals. By constructing a label association graph to sort out the inherent relationships between labels, and dynamically adjusting the label embedding vector in combination with changes in data distribution, the use of label information is more comprehensive, avoiding the information waste caused by isolated label processing in traditional methods. For cases where the data category distribution of different clients is inconsistent, a data imbalance coefficient is introduced to quantify this difference and is added to the training of local models, enabling local models to adapt to the data characteristics of different terminals and solving the impact of uneven data distribution on training results in distributed scenarios. In the feature processing stage, the optimized label features are fully combined with image features, highlighting the role of key features, allowing local models to learn more comprehensive image information and improving the recognition accuracy problem caused by insufficient feature fusion. Finally, when aggregating the global model, it is grouped according to the parameter similarity of the local models of each client and the aggregation method is adjusted, so that the global model can better absorb the training results of each client, reducing the parameter deviation that may occur in a single aggregation mode, and ensuring that the global model maintains stable performance in different application scenarios. These optimizations work together, forming a complete optimization process from label processing, data adaptation, feature fusion to the aggregation of local models. This not only meets the requirements of distributed data privacy protection but also fits the task characteristics of multi-label image recognition, ultimately achieving a simultaneous improvement in model recognition performance and applicability. Attached Figure Description

[0023] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a schematic diagram illustrating the overall process of constructing a multi-label image recognition model based on federated learning, according to an embodiment of the present invention. Detailed Implementation

[0025] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.

[0026] Example 1, referring to Figure 1 As an embodiment of the present invention, a method for constructing a multi-label image recognition model based on federated learning is provided, including steps S100 to S500: S100. Obtain the multi-label image dataset of each client in the federated learning system, and perform label embedding processing on the multi-label image dataset to obtain the mask label embedding vector of each client.

[0027] S200. Construct a label association graph based on the masked label embedding vector, and extract features from the label association graph using a graph attention autoencoder to obtain label encoding features.

[0028] S300: Each client extracts image features from the multi-label image dataset, fuses the label-encoded features with the image features, and inputs them into the classifier for training to obtain a local model.

[0029] S400: Each client calculates the imbalance coefficient for each category based on the number of samples in each category of the multi-label image dataset, and uploads the model parameters and imbalance coefficients of the local model to the server.

[0030] The S500 server uses a clustering algorithm to divide local model parameters into multiple model groups, and aggregates parameters within each model group to obtain multiple aggregated models. It calculates the average value of the imbalance coefficient of the client within each model group, and performs weighted aggregation on each aggregated model based on the average value to obtain the global model.

[0031] It should be noted that multi-label image recognition needs to take into account both the correlation and dependence between labels and the matching of image features. Traditional models often ignore the label correlation, resulting in inaccurate feature representation. In federated learning scenarios, the multi-label image datasets of each client have problems such as class imbalance and heterogeneous distribution. Moreover, sharing the original data may cause privacy leakage risks. The traditional federated learning model aggregation method does not consider the class imbalance coefficient, which may cause the global model performance to be biased towards the class with sufficient samples. It is difficult to adapt to the actual needs of multi-label image recognition. Therefore, it is crucial to build a federated learning image recognition model that adapts to the characteristics of multi-label.

[0032] Therefore, to address the aforementioned issues such as insufficient label association mining, imbalanced client data, and difficulty in balancing privacy and model performance, steps S100-S500 are used to embed labels and extract association graph features from multi-label datasets. Image and label features are fused to train local models, and the imbalance coefficient is used to cluster and weight the local model parameters. This approach fully utilizes the association characteristics between multiple labels to improve feature representation accuracy, adapts to the heterogeneity and imbalance of data under federated learning, and ensures data privacy and security, thus realizing the construction of a global model for multi-label image recognition.

[0033] Example 2, refer to Figure 1 This is one embodiment of the present invention. Based on the above embodiment, a method for constructing a multi-label image recognition model based on federated learning is provided.

[0034] In this embodiment of the application, S100, the multi-label image dataset of each client in the federated learning system is obtained, and the multi-label image dataset is processed by label embedding to obtain the mask label embedding vector of each client. In this embodiment, the federated learning system comprises K clients, each possessing a local multi-label image dataset. Taking a smart grid safety monitoring scenario as an example, assume the federated learning system includes three clients, each corresponding to a substation in a different region: the first client's multi-label image dataset contains 1000 images, covering 10 categories including "not wearing a safety helmet," "violation of operating procedures," and "smoking"; the second client's multi-label image dataset contains 800 images, covering the same 10 label categories; and the third client's multi-label image dataset contains 1200 images. Due to differences in the working environment across different regions, the category distribution of each client is unbalanced. For example, the first client has 300 samples labeled "not wearing a safety helmet," while the "smoking" label has only 20 samples.

[0035] The steps involved in performing label embedding processing on the multi-label image dataset to obtain the mask label embedding vectors for each client include A1~A3: A1. Encode the labels in the multi-label image dataset, generate label embedding vectors, and initialize the state embedding vectors based on the true state of the labels; In this embodiment, a pre-trained CLIP model text encoder is used to encode the tags. First, the text descriptions of 10 tags, such as "not wearing a safety helmet", "violation of regulations", and "smoking", are input into the text encoder to generate a tag embedding vector set L. This tag embedding vector set contains 10 tag embedding vectors, and each tag embedding vector has a dimension of 512. Taking the first label "not wearing a helmet" as an example, the corresponding label embedding vector is denoted as L1, which is a 512-dimensional vector; the label embedding vector corresponding to the third label "smoking" is denoted as L3; at the same time, the state embedding vector set S is initialized based on the actual state of the label. The state embedding vector set also contains 10 state embedding vectors, each with a dimension of 512. During initialization, the settings are made according to the actual labeling of the image: if the first label "not wearing a helmet" exists in the image, the corresponding state embedding vector S1 is initialized as a 512-dimensional positive sample vector, where each dimension element is set to 1; if the third label "smoking" is missing in the image, the corresponding state embedding vector S3 is initialized as a 512-dimensional negative sample vector, where each dimension element is set to 0.

[0036] A2. During the training iteration, predictions are made on the multi-label image dataset to obtain the prediction confidence of each label, and the state embedding vector is updated according to the relationship between the prediction confidence and the preset threshold. Taking an image as an example, the prediction confidence scores of 10 labels for the image are predicted and denoted as p1, p2, ..., p10. The prediction confidence score for the first label, "not wearing a helmet," is p1 = 0.82; for the second label, "violation of regulations," it is p2 = 0.45; and for the third label, "smoking," it is p3 = 0.52. In this embodiment, a preset threshold τ = 0.5 and a tolerance range ε ​​= 0.1 are set. When the prediction confidence score of a label is within the interval [τ-ε, τ+ε], that is, within the range [0.4, 0.6]... Within the threshold range, the label is determined to be in an unknown state. In this example, the prediction confidence of the second label, p2=0.45, is within the threshold range, so the corresponding state embedding vector S2 is updated to a 512-dimensional unknown state vector, where each dimension element is set to -1. The prediction confidence of the third label, p3=0.52, is also within the threshold range, so the state embedding vector S3 is updated to an unknown state vector, where each dimension element is set to -1. The prediction confidence of the first label, p1=0.82, is not within the threshold range, so the initial value of the state embedding vector S1 remains unchanged. Through the above processing, the updated set of state embedding vectors S' is obtained, reflecting the degree of certainty of each label in the current prediction state.

[0037] A3. Perform element-wise multiplication between the label embedding vector and the updated state embedding vector to obtain the mask label embedding vector. In this embodiment, each tag embedding vector in the tag embedding vector set L is multiplied element-wise with the corresponding state embedding vector in the updated state embedding vector set S'. Taking the first tag "not wearing a helmet" as an example, the mask tag embedding vector is calculated as L1×S'1, where × represents the element-wise multiplication operation, that is, multiplying the first element of the tag embedding vector L1 with the first element of the state embedding vector S'1, multiplying the second element of L1 with the second element of S'1, and so on until the 512th dimension, resulting in a 512-dimensional mask tag embedding vector. Since the state embedding vector S'1 retains the initial positive sample vector unchanged in step A2, that is, each element of each dimension is 1, the calculated mask tag embedding vector is L1×1=L1, which completely preserves the original semantic information of the tag "not wearing a helmet"; for the second tag "violation", the mask tag embedding vector is calculated as L2× S'2, since the state embedding vector S'2 is updated to an unknown state vector in step A2, that is, all dimensions are -1, the calculated mask label embedding vector is L2×(-1)=-L2. The negative value indicates that the label "violation" is in an uncertain state. For the third label "smoking", the calculated mask label embedding vector is L3×S'3. Similarly, since the state embedding vector S'3 is updated to -1, the mask label embedding vector is L3×(-1)=-L3. The same processing is performed on 10 labels in sequence, and finally the mask label embedding vector set of each client is obtained. This mask label embedding vector set contains 10 mask label embedding vectors, which integrate the semantic features and dynamic state information of the labels.

[0038] In an optional implementation, the label embedding processing of the multi-label image dataset in step S100 to obtain the masked label embedding vectors for each client can also be based on a label co-occurrence statistical processing method. Specifically, each client encodes the labels in the multi-label image dataset to generate a set of label embedding vectors; simultaneously, the co-occurrence frequency among the labels in the multi-label image dataset is counted to construct a label co-occurrence matrix. Taking the first label "not wearing a safety helmet" and the second label "violation" as an example, the number of times the two labels appear simultaneously in the same image is counted. Assuming they co-occur 180 times in 1000 images, the co-occurrence frequency is 0.18. Then, the state embedding vector is initialized based on the label co-occurrence matrix, and the co-occurrence frequency vector of each label is used as the state embedding vector. The label embedding vector and the corresponding state embedding vector are weighted and summed, where the weight of the label embedding vector is set to 0.7 and the weight of the state embedding vector is set to 0.3, resulting in the masked label embedding vector. This method enhances the expressive power of label embedding by utilizing the co-occurrence relationship between labels.

[0039] In this embodiment of the application, S200, a label association graph is constructed based on the mask label embedding vector, and the label association graph is subjected to feature extraction by a graph attention autoencoder to obtain label encoding features; In this embodiment, the label association graph uses labels as nodes and the similarity between labels as the edge weights. The masked label embedding vector set obtained in step S100 contains 10 masked label embedding vectors, each with a dimension of 512. By calculating the similarity between the masked label embedding vectors, a 10×10 adjacency matrix is ​​constructed to represent the connection relationship of the label association graph. Subsequently, the graph attention autoencoder processes the label association graph through two components: an encoder and a decoder. The encoder is responsible for extracting the correlation features between labels, and the decoder is responsible for reconstructing the original label features to verify the encoding quality.

[0040] The steps for extracting label encoding features from the label association graph using a graph attention autoencoder include B1 to B3: B1. Calculate the similarity between each label node in the label association graph based on the dot product operation between the masked label embedding vectors, and normalize the similarity. Use the normalized similarity as the edge weight of the label association graph. In this embodiment, taking the first label "not wearing a safety helmet" and the second label "violation" as examples, the elements of the corresponding dimensions of the mask label embedding vectors of the two labels are multiplied and then summed. That is, the elements of the first dimension are multiplied, the elements of the second dimension are multiplied, and so on until the 512th dimension, to obtain the dot product similarity value. The dot product similarity between each of the 10 labels is calculated sequentially to obtain a 10×10 similarity matrix. The similarity matrix is ​​normalized by dividing each similarity value by the maximum value of all similarity values, so that the normalized similarity value is within the interval [0, 1]. For example, if the maximum value in the similarity matrix is ​​200.5, and the similarity between two labels is 150.2, then the normalized similarity is 150.2 ÷ 200.5 ≈ 0.75. The normalized 10×10 similarity matrix is ​​used as the adjacency matrix of the label association graph, where each element represents the weight of the edge between the corresponding label nodes.

[0041] B2. Calculate the attention coefficient of each tag node based on similarity, and then perform weighted aggregation of the features of other tag nodes connected to the current tag node according to the attention coefficient to obtain the attention features of each tag node. In this embodiment, taking the first label "not wearing a helmet" as an example, the normalized similarity of this label node with the other 9 label nodes are 0.75, 0.62, 0.45, etc. These normalized similarities are then subjected to softmax normalization, i.e., each similarity value is exponentially divided by the sum of all exponential values ​​to obtain the attention coefficient. Assuming the attention coefficient of the first label to the second label is 0.18, and the attention coefficient to the third label is 0.15, with the sum of all attention coefficients being 1, the features of the other label nodes are weighted and aggregated based on the attention coefficients. The masked label embedding vector of the second label is multiplied by 0.18, the masked label embedding vector of the third label is multiplied by 0.15, and so on, until all weighted vectors are summed to obtain the attention feature of the first label. The attention features of the other labels are calculated in the same way, ultimately resulting in 10 attention features of 512 dimensions.

[0042] B3. The attention features are decoded and reconstructed using the decoder of the graph attention autoencoder, and the encoder of the graph attention autoencoder is trained based on the difference between the original features and the reconstructed features to obtain the label encoded features. The 10 attention features obtained in step B2 are input into the decoder for decoding and reconstruction. In this embodiment, the decoder adopts a multi-layer neural network structure, mapping the attention features back to the original mask label embedding vector space layer by layer, and outputting 10 512-dimensional reconstructed feature vectors. The difference between the original mask label embedding vector and the reconstructed feature vector is calculated using mean squared error, i.e., the sum of the squares of the differences in corresponding dimensions is calculated and averaged, and then the reconstruction losses of the 10 labels are summed to obtain the total reconstruction loss. Finally, based on the total reconstruction loss, the encoder of the graph attention autoencoder is trained using the backpropagation algorithm, continuously adjusting the encoder's weight parameters to gradually reduce the reconstruction loss. After multiple rounds of training iterations, the attention features obtained in step B2 become the label encoding features, which contain 10 512-dimensional feature vectors, integrating semantic associations and co-occurrence patterns between labels.

[0043] In an optional implementation, in step S200, feature extraction of the label association graph is performed using a graph attention autoencoder to obtain label encoding features. Alternatively, a multi-layer graph convolutional network can be used for feature extraction. Specifically, after constructing the label association graph based on the masked label embedding vector, the masked label embedding vector is used as the initial feature input to the multi-layer graph convolutional network. The graph convolutional network contains three graph convolutional layers. Each graph convolutional layer aggregates and transforms the node features according to the adjacency matrix of the label association graph. Taking the first graph convolutional layer as an example, for the first label node, the features of neighboring nodes are weighted and summed according to the edge weights between this label node and other label nodes. Then, a linear transformation is performed using a learnable weight matrix, and the updated features are obtained after passing through an activation function. This process is repeated through three graph convolutional layers to obtain label node features that incorporate multi-hop neighborhood information. The features output by the last graph convolutional layer are used as the label encoding features. These features contain ten 512-dimensional feature vectors, reflecting the association patterns between labels.

[0044] In this embodiment of the application, in S300, each client extracts image features from the multi-label image dataset, and then fuses the label-encoded features with the image features and inputs them into the classifier for training to obtain a local model; Each client performs fusion processing based on the label encoding features obtained in step S200, combined with image features extracted from the multi-label image dataset. In this embodiment, the classifier adopts a two-stage architecture, including a feature transformation network and a fully connected classification layer. The feature transformation network is responsible for performing deep semantic transformation on the fused features, and the fully connected classification layer is responsible for outputting the prediction results of each label. During training, a joint loss function containing three loss terms is used to ensure that the local model maintains parameter consistency with the global model while optimizing classification performance.

[0045] The steps involved in fusing label-encoded features with image features and then inputting the fused features into a classifier for training to obtain a local model include C1 to C3: C1. Flatten the label-encoded features into a one-dimensional feature vector, and concatenate the one-dimensional feature vector with the image features extracted from the multi-label image dataset to obtain the fused features; In this embodiment, the 10 feature vectors obtained in step S200 are first flattened into one-dimensional feature vectors, that is, the 10 512-dimensional vectors are sequentially concatenated into a 5120-dimensional one-dimensional vector. Simultaneously, a convolutional neural network is used to extract image features from the multi-label image dataset. Taking an input image with a resolution of 224×224 as an example, after multiple convolution and pooling operations of the convolutional neural network, a 2048-dimensional image feature vector is obtained. The 5120-dimensional label feature vector is concatenated with the 2048-dimensional image feature vector to obtain a 7168-dimensional fused feature vector. This fused feature vector simultaneously contains semantic association information of the labels and visual feature information of the image.

[0046] C2. The fused features are input into the feature transformation network in the classifier for feature transformation, and then the prediction results of each label are output through the fully connected classification layer of the classifier. The 7168-dimensional fused features obtained in step C1 are input into the feature transformation network in the classifier. In this embodiment, the feature transformation network adopts a multi-layer fully connected neural network structure, containing three hidden layers with the number of neurons in the hidden layers set to 4096, 2048, and 1024, respectively. The fused features pass through these three hidden layers sequentially, with each hidden layer followed by a ReLU activation function for non-linear transformation. After passing through the feature transformation network, a 1024-dimensional transformed feature is obtained. This transformed feature is then input into the fully connected classification layer, which contains 10 output neurons, corresponding to 10 label categories. The output of the fully connected classification layer is processed by a sigmoid activation function to obtain the prediction confidence of each label. Taking a certain image as an example, the prediction result shows that the confidence of the first label "not wearing a safety helmet" is 0.85, and the confidence of the second label "violation" is 0.32. The prediction results for the 10 labels are output sequentially.

[0047] C3. Construct the loss function, which includes a classification loss term, a reconstruction loss term, and a regularization loss term, and train the local model for each client based on the loss function; In this embodiment, a joint loss function comprising three loss terms is constructed. The classification loss term employs a weighted cross-entropy loss, with a weight coefficient calculated for each label category based on its sample size. Categories with fewer samples are assigned higher weights to mitigate class imbalance. The weight coefficients are calculated as follows: the total number of samples is divided by the product of the total number of categories and the number of samples in that category. Taking the first client as an example, the number of samples for the "not wearing a helmet" label is 300, and the corresponding weight coefficient is calculated as 1000÷(10×300)≈0.33. The number of samples for the "smoking" label is 20, and the corresponding weight coefficient is calculated as 1000÷(10×20)=5.0. The reconstruction loss term is the reconstruction loss of the graph attention autoencoder calculated in step B3, which is used to constrain the label encoding features to retain the correlation information between labels. The regularization loss term is used to constrain the consistency between the local model parameters and the global model parameters, and the squared Euclidean norm of the difference between the local model parameters and the global model parameters is calculated. Based on the loss function constructed by the above three losses, the parameters of the local model are updated by the gradient descent algorithm. After multiple rounds of iterative training, each client obtains the optimized local model.

[0048] The loss function is specifically expressed as: ; In the formula, Loss is the total loss. For classifying losses, The reconstruction loss is the difference between the original features and the reconstructed features, and w represents the parameters of the local model. These are the parameters of the global model. The regularization coefficient is . The squared Euclidean norm of the difference between the local model parameters and the global model parameters; In this embodiment, we will use a training iteration of the first client as an example for illustration. The classification loss of this client... The calculated value of 2.35 indicates a certain deviation between the current local model's predictions of each label and the actual annotations; reconstruction loss. The value is 18.3, derived from the total reconstruction loss calculated during the graph attention autoencoder training process in step B3; the regularization coefficient μ is set to 0.015. The local model parameters w contain all trainable parameters in the classifier, assuming a total of 5 million parameters. The global model parameters... For the parameters aggregated from the previous round of federated learning, the difference between the local model parameters and the global model parameters is calculated. This involves subtracting each of the 5 million parameters one by one and then calculating the sum of squared differences. Assuming the squared Euclidean norm is 1200, the regularization loss term is 0.015 × 1200 = 18, and the total loss is calculated as 2.35 + 18.3 + 18 = 38.65. Finally, based on the calculated total loss, the local model parameters are updated using the backpropagation algorithm to reduce the total loss value and allow the local model to gradually converge.

[0049] In an optional implementation, the fusion of label-encoded features and image features in step S300, followed by inputting the fused features into the classifier for training, can also employ an attention-based fusion method to obtain the local model. Specifically, each label feature vector in the label-encoded features is interactively calculated with the image features, and the attention weight of each label for different regions of the image is learned through the attention mechanism. Taking the first label "not wearing a safety helmet" as an example, the attention score between the label feature vector and the image feature vector is calculated, and the image features are weighted according to the attention score to obtain a customized image representation for the label. Customized image representations are generated for each of the 10 labels in sequence, and these customized image representations are added element-wise with the label-encoded features to obtain fused features. The fused features are then input into the classifier for training, and the classifier outputs the prediction results for each label. Next, a loss function containing a classification loss term and a reconstruction loss term is constructed, where the classification loss term calculates the weight coefficients based on the number of samples in each category and is weighted, and the reconstruction loss term is the reconstruction loss of the graph attention autoencoder. Finally, the local model is trained based on the loss function.

[0050] In this embodiment of the application, S400, each client calculates the imbalance coefficient of each category based on the number of samples of each category in the multi-label image dataset, and uploads the model parameters and imbalance coefficients of the local model to the server; In this embodiment, after each client completes local model training, it needs to calculate the imbalance coefficients for each category to reflect the category distribution characteristics of the local data: the larger the imbalance coefficient, the fewer the samples in that category, and the more attention should be paid to it in the subsequent global model aggregation process. Taking the first client as an example, its multi-label image dataset contains 1000 images involving 10 label categories. The occurrence frequency of each label category in the dataset is counted sequentially to obtain the sample quantity of each category: for example, the "not wearing a helmet" label appears 300 times, the "violation" label appears 150 times, and the "smoking" label appears 20 times. The imbalance coefficient is calculated based on the total number of samples, the total number of categories, and the sample quantity of each category. After the calculation is completed, the client packages all the parameters of the trained local model and the calculated imbalance coefficients for the 10 categories together and uploads them to the server.

[0051] The step of calculating the imbalance coefficient for each category based on the number of samples in each category of the multi-label image dataset includes counting the total number of samples and the number of samples in each category of the multi-label image dataset, and then calculating the imbalance coefficient. The imbalance coefficient is expressed as follows: ; In the formula, Let be the imbalance coefficient for category c. Where C is the total number of samples and C is the total number of categories. The number of samples in category c; In this embodiment, the calculation of the imbalance coefficients for each category by the first client is used as an example for illustration. The total number of samples for this client is... The sample size is 1000, and the total number of categories C is 10. This represents the sample size for the "not wearing a helmet" label. The value is 300. Substituting this value into the formula, we obtain the imbalance coefficient. =1000÷(10×300)≈0.33; For the "violation operation" label, its sample size... The unbalance coefficient is calculated to be 150. =1000÷(10×150)≈0.67; For the "smoking" label, the sample size is... The value is 20, and the imbalance coefficient is calculated. =1000÷(10×20)=5.0. It can be seen that the smaller the sample size of the category, the larger the imbalance coefficient. The "smoking" label, with only 20 samples, has an imbalance coefficient of 5.0, which is much higher than the 0.33 of the "not wearing a helmet" label with sufficient samples. Calculating the imbalance coefficients of the 10 categories in turn yields the imbalance coefficient vector {0.33, 0.67, 5.0, ...}, which reflects the degree of imbalance in the category distribution of the first client's local data.

[0052] In the embodiments of this application, S500, the server divides the local model parameters into multiple model groups through a clustering algorithm, and performs parameter aggregation within each model group to obtain multiple group aggregated models. The average value of the imbalance coefficient of the client within each model group is calculated, and the group aggregated models are weighted and aggregated based on the average value to obtain the global model. In this embodiment, after receiving the local model parameters and imbalance coefficients uploaded by three clients, the server employs a two-stage aggregation strategy to construct a global model: The first stage involves clustering based on the similarity of the local model parameters, grouping clients with similar parameter distributions into the same model group, and aggregating parameters within each group to obtain a group-aggregated model; the second stage involves weighted aggregation of the group-aggregated models based on the average imbalance coefficient of the clients within each model group, resulting in the final global model. This two-stage aggregation strategy considers both the parameter differences between clients and the imbalanced category distribution characteristics of each client, ensuring that the global model can adapt to the data characteristics of different clients.

[0053] The steps involved in dividing local model parameters into multiple model groups using a clustering algorithm and then aggregating parameters within each model group to obtain aggregated models from multiple groups include D1 to D2: D1. Extract the parameters of the classifiers in each local model as clustering features, and divide each local model into multiple model groups based on the similarity between the parameters of the classifiers using a clustering algorithm; In this embodiment, the server extracts classifier parameters as clustering features from the local models of the three clients. The classifier includes a feature transformation network and a fully connected classification layer, and its parameters account for the majority of the total parameters in the local model. Taking the first client as an example, the classifier parameters contain approximately 2 million parameter values. These parameters are flattened into a 2 million-dimensional vector as the clustering feature vector for that client. Similarly, the classifier parameters of the second and third clients are extracted to obtain three clustering feature vectors. Then, the cosine similarity between the three clustering feature vectors is calculated, which is calculated by dividing the dot product of two clustering feature vectors by the magnitude of each vector. Assuming the classifier parameter similarity between the first and second clients is 0.85, the similarity between the first and third clients is 0.42, and the similarity between the second and third clients is 0.38, the K-means clustering algorithm is used, with a cluster size of 2, to divide the local models of the three clients into two model groups. Since the parameter similarity between the first and second clients is high, they are grouped into the first model group; the third client has a low similarity with the first two, so it is grouped into the second model group.

[0054] D2. Calculate the weight coefficients based on the similarity between the local model parameters within each model group, and then sum the local model parameters within each model group according to the weight coefficients to obtain the group aggregation model corresponding to each model group. In this embodiment, taking the first model group as an example, this model group includes the local models of the first client and the second client. The parameter similarity between the two, calculated in step D1, is 0.85. After normalization, the weight coefficients for both clients are 0.5. A weighted sum is performed on all parameters of the two local models according to the weight coefficients. Specifically, each parameter value of the first client's local model is multiplied by 0.5, and the corresponding parameter value of the second client's local model is multiplied by 0.5, then summed to obtain the aggregated parameter value. This same process is applied to all parameters sequentially to obtain the group aggregate model for the first model group. For the second model group, since it only includes the third client, the parameters of the group aggregate model are directly equal to the parameters of the third client's local model. Thus, two group aggregate models are obtained. The server calculates the average imbalance coefficient of the clients within each model group and performs weighted aggregation on the group aggregate models based on the average imbalance coefficient to obtain the global model.

[0055] In one optional implementation, step S500, which divides the local model parameters into multiple model groups using a clustering algorithm and aggregates parameters within each model group to obtain multiple aggregated models, can also employ a reference model-based aggregation method. Specifically, the server first uses the global model obtained from the previous federated learning iteration as the reference model; if it is the first iteration, a randomly initialized model is used as the reference model. The parameters of the classifier in each client's local model are extracted, and the similarity between the parameters of each local model and the parameters of the reference model is calculated. Based on this similarity, the local models are divided into multiple model groups using a clustering algorithm. When aggregating parameters within each model group, a weight coefficient is calculated based on the similarity between the parameters of each local model and the parameters of the reference model. Local models with higher similarity receive greater weights. The local model parameters within each model group are weighted and summed according to the weight coefficients to obtain the aggregated model corresponding to each model group. This method uses the reference model as a benchmark for aggregation, ensuring that the parameter distribution of the aggregated model is closer to the overall optimization direction.

[0056] In summary, based on the federated learning framework, targeted optimizations were made to meet the practical needs of multi-label image recognition. Without compromising the privacy of distributed terminal data, the overall performance of the global model was improved: by constructing a label association graph to clarify the inherent relationships between labels, and dynamically adjusting the label embedding vectors in conjunction with changes in data distribution, the utilization of label information is more comprehensive, avoiding the information waste caused by isolated label processing in traditional methods; for situations where the data category distributions of different clients are inconsistent, a data imbalance coefficient is introduced to quantify this difference and is added to the training of local models, enabling local models to adapt to the data characteristics of different terminals and solving the impact of uneven data distribution on training results in distributed scenarios; in the feature processing stage, optimized label features are fully combined with image features, highlighting the role of key features, allowing local models to learn more comprehensive image information and improving the recognition accuracy problem caused by insufficient feature fusion; finally, during global model aggregation, grouping is based on the parameter similarity of local models from each client, and the aggregation method is adjusted so that the global model can better absorb the training results from each client, reducing parameter bias that may occur in a single aggregation mode, and ensuring that the global model maintains stable performance in different application scenarios. These optimizations work together, forming a complete optimization process from label processing, data adaptation, feature fusion to the aggregation of local models. This not only meets the requirements of distributed data privacy protection but also fits the task characteristics of multi-label image recognition, ultimately achieving a simultaneous improvement in model recognition performance and applicability.

[0057] Example 3 illustrates a method for constructing a multi-label image recognition model based on federated learning. It should be noted that the technical solution of this system for constructing a multi-label image recognition model based on federated learning is based on the same concept as the method described above. Details not described in detail in the system for constructing a multi-label image recognition model based on federated learning in this example can be found in the description of the method described above.

[0058] This embodiment also provides a system for constructing a multi-label image recognition model based on federated learning, including: Data Acquisition and Label Embedding Module: This module is used to acquire multi-label image datasets from each client in the federated learning system, and to perform label embedding processing on the multi-label image datasets to obtain the masked label embedding vectors for each client. Feature extraction module: used to construct a label association graph based on masked label embedding vectors, and to extract features from the label association graph through a graph attention autoencoder to obtain label encoded features; Local model training module: This module is used by each client to extract image features from a multi-label image dataset, fuse the label-encoded features with the image features, and then input the fused features into a classifier for training to obtain a local model. Parameter calculation and upload module: This module is used by each client to calculate the imbalance coefficient of each category based on the number of samples of each category in the multi-label image dataset, and upload the model parameters and imbalance coefficients of the local model to the server. Global model aggregation module: The server uses a clustering algorithm to divide local model parameters into multiple model groups, aggregates parameters within each model group to obtain multiple aggregated models, calculates the average imbalance coefficient of clients within each model group, and performs weighted aggregation of each aggregated model group based on the average value to obtain the global model.

[0059] This embodiment also provides an electronic device applicable to the construction of a multi-label image recognition model based on federated learning, comprising: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the method for constructing a multi-label image recognition model based on federated learning as proposed in the above embodiment.

[0060] This embodiment also provides a storage medium storing a computer program that, when executed by a processor, implements the method for constructing a multi-label image recognition model based on federated learning as proposed in the above embodiments.

[0061] The storage medium proposed in this embodiment and the method for constructing a multi-label image recognition model based on federated learning proposed in the above embodiments belong to the same inventive concept. Technical details not described in detail in this embodiment can be found in the above embodiments, and this embodiment has the same beneficial effects as the above embodiments.

[0062] Based on the above description of the implementation methods, those skilled in the art can clearly understand that the present invention can be implemented using software and necessary general-purpose hardware, and of course, it can also be implemented using hardware. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk, or optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of the various embodiments of the present invention.

[0063] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for constructing a multi-label image recognition model based on federated learning, characterized in that, include: Obtain the multi-label image dataset of each client in the federated learning system, and perform label embedding processing on the multi-label image dataset to obtain the mask label embedding vector of each client. A label association graph is constructed based on the masked label embedding vector, and the label encoding features are extracted from the label association graph by a graph attention autoencoder. Each client extracts image features from a multi-label image dataset, and then fuses the label-encoded features with the image features and inputs them into a classifier for training to obtain a local model. Each client calculates the imbalance coefficient for each category based on the number of samples in each category of the multi-label image dataset, and uploads the model parameters and imbalance coefficients of the local model to the server; The server uses a clustering algorithm to divide local model parameters into multiple model groups, and aggregates parameters within each model group to obtain multiple aggregated models. It calculates the average value of the imbalance coefficient of the client within each model group, and performs weighted aggregation on each aggregated model based on the average value to obtain the global model.

2. The method for constructing a multi-label image recognition model based on federated learning as described in claim 1, characterized in that, The steps for performing label embedding processing on the multi-label image dataset to obtain the mask label embedding vectors for each client include: The labels in the multi-label image dataset are encoded to generate label embedding vectors, and the state embedding vectors are initialized based on the true state of the labels. During the training iteration, predictions are made on the multi-label image dataset to obtain the prediction confidence of each label, and the state embedding vector is updated according to the relationship between the prediction confidence and a preset threshold. The label embedding vector is multiplied element-wise with the updated state embedding vector to obtain the mask label embedding vector.

3. The method for constructing a multi-label image recognition model based on federated learning as described in claim 2, characterized in that, The steps for extracting label encoding features from the label association graph using a graph attention autoencoder include: The similarity between each label node in the label association graph is calculated based on the dot product operation between the mask label embedding vectors, and the similarity is normalized and used as the edge weight of the label association graph. The attention coefficient of each tag node is calculated based on the similarity, and the features of other tag nodes connected to the current tag node are weighted and aggregated according to the attention coefficient to obtain the attention features of each tag node. The attention features are decoded and reconstructed using the decoder of the graph attention autoencoder, and the encoder of the graph attention autoencoder is trained based on the difference between the original features and the reconstructed features to obtain the label encoded features.

4. The method for constructing a multi-label image recognition model based on federated learning as described in claim 3, characterized in that, The steps of fusing the label-encoded features with image features and then inputting the fused features into a classifier for training to obtain a local model include: The label-encoded features are flattened into a one-dimensional feature vector, and the one-dimensional feature vector is concatenated with the image features extracted from the multi-label image dataset to obtain the fused features; The fused features are input into the feature transformation network in the classifier for feature transformation, and then the prediction results of each label are output through the fully connected classification layer of the classifier. A loss function is constructed, which includes a classification loss term, a reconstruction loss term, and a regularization loss term, and a local model for each client is trained based on the loss function.

5. The method for constructing a multi-label image recognition model based on federated learning as described in claim 4, characterized in that, The loss function is specifically expressed as follows: ； In the formula, Loss is the total loss. For classifying losses, The reconstruction loss is the difference between the original features and the reconstructed features, and w represents the parameters of the local model. These are the parameters of the global model. The regularization coefficient is . It is the squared Euclidean norm of the difference between the local model parameters and the global model parameters.

6. The method for constructing a multi-label image recognition model based on federated learning as described in claim 5, characterized in that, The steps for calculating the imbalance coefficient of each category based on the number of samples in each category of a multi-label image dataset include: The total number of samples and the number of samples in each category of the multi-label image dataset are counted, and the imbalance coefficient is calculated. The imbalance coefficient is expressed as follows: ； In the formula, Let be the imbalance coefficient for category c. Where C is the total number of samples and C is the total number of categories. The number of samples in category c.

7. The method for constructing a multi-label image recognition model based on federated learning as described in claim 6, characterized in that, The steps involved in dividing local model parameters into multiple model groups using a clustering algorithm, and then aggregating parameters within each model group to obtain aggregated models from multiple groups, include: The parameters of the classifiers in each local model are extracted as clustering features, and based on the similarity between the parameters of the classifiers, each local model is divided into multiple model groups by a clustering algorithm; The weight coefficients are calculated based on the similarity between the local model parameters within each model group, and the local model parameters within each model group are weighted and summed according to the weight coefficients to obtain the group aggregation model corresponding to each model group.

8. A system for constructing a multi-label image recognition model based on federated learning, using the method described in any one of claims 1-7, characterized in that, include: Data acquisition and label embedding module: used to acquire multi-label image datasets from each client in the federated learning system, and to perform label embedding processing on the multi-label image datasets to obtain the mask label embedding vectors of each client; Feature extraction module: used to construct a label association graph based on the mask label embedding vector, and to extract features from the label association graph through a graph attention autoencoder to obtain label encoding features; Local model training module: This module is used by each client to extract image features from a multi-label image dataset, fuse the label-encoded features with the image features, and then input the fused features into a classifier for training to obtain a local model. Parameter calculation and upload module: used by each client to calculate the imbalance coefficient of each category based on the number of samples of each category in the multi-label image dataset, and upload the model parameters and imbalance coefficients of the local model to the server; Global model aggregation module: The server uses a clustering algorithm to divide local model parameters into multiple model groups, aggregates parameters within each model group to obtain multiple aggregated models, calculates the average value of the imbalance coefficient of the client within each model group, and performs weighted aggregation on each aggregated model group based on the average value to obtain the global model.

9. An electronic device, comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, they implement the steps of the method for constructing a multi-label image recognition model based on federated learning as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the method for constructing a multi-label image recognition model based on federated learning as described in any one of claims 1 to 7.

Images