A Collaborative Training Method for Edge Cloud in Few-Sample Object Detection Based on Fixed Class Prototypes and Peer Evaluation-Reflection Mechanism

Abstract

This invention belongs to the fields of computer vision, edge computing, and federated learning, and relates to a cloud-based collaborative training method for few-shot object detection based on fixed class prototypes and a mutual evaluation-reflection mechanism. This method constructs class prototypes in the cloud and designs prototype alignment losses, while only extracting shallow features at the edge and uploading them, reducing computational, storage, and communication overhead. It suppresses oscillations in few-shot training of new classes through reflective loss and hard example replay. By jointly weighting edge mutual evaluation and data quality, it dynamically adjusts the global fusion weights and generates personalized training configurations, thereby improving convergence stability, detection accuracy, and model generalization ability.

Description

Technical Field

[0001] This invention relates to the fields of computer vision, edge computing, and federated learning, and in particular to a collaborative training method for edge cloud for few-shot object detection based on fixed class prototypes and a mutual evaluation-reflection mechanism. Background Technology

[0002] Few-shot object detection aims to achieve good detection performance even with a limited number of samples of new categories. Most existing few-shot object detection methods follow the traditional object detection framework, adding and fine-tuning a classifier for the new category to the detection head to adapt to the feature distribution of that new category. These methods typically require updating a large-scale classifier weight matrix containing numerous parameters, resulting in high deployment and online update costs for edge servers with limited computing and storage resources.

[0003] With the synergistic development of edge computing and cloud computing, deploying the shallow network of a model at the edge to complete feature extraction and uploading intermediate features to the cloud for complex inference and training has become an effective solution that balances real-time performance and computing power utilization. However, in scenarios with few samples, new category samples are scarce and their distribution is unstable. If the method of frequently updating classifier weights is still used, it is easy to cause oscillations in the training process, making it difficult for the model to converge, and the parameter synchronization overhead between the edge and the cloud is relatively large.

[0004] On the other hand, existing federated learning or edge collaborative training typically employs a simple model parameter averaging strategy to construct the global model, rarely considering the differences in data distribution, sample quality, and model generalization ability among different edge servers. Furthermore, it lacks mechanisms for mutual evaluation and learning among edge servers, resulting in insufficient adaptability of the global model to certain edge scenarios. In addition, traditional methods rarely explicitly utilize feature statistics across rounds, lack constraints on the feature stability of new categories, and are prone to performance instability in new class detection tasks with few samples.

[0005] Therefore, it is necessary to design a few-shot object detection training method suitable for edge-cloud collaborative architecture. While reducing the burden of parameter updates on edge servers, this method introduces more refined feature constraints and cross-edge collaborative mechanisms to improve the generalization ability of the global model on multi-source heterogeneous data and enhance its ability to quickly adapt to and stably learn new categories. Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings of existing few-shot target detection methods, such as high resource requirements on the edge side, large overhead of classifier weight updates, coarse global model fusion, and unstable new class features. This invention proposes a few-shot target detection edge cloud collaborative training method based on fixed class prototypes and mutual evaluation-reflection mechanism. By replacing large-scale trainable classifiers with fixed class prototypes, introducing cross-round feature reflection mechanism and edge node mutual evaluation-dynamic fusion mechanism, this invention achieves efficient, stable, and highly generalizable few-shot target detection model training.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A collaborative training method for edge cloud detection of few-shot targets based on fixed class prototypes and mutual evaluation-reflection mechanism includes the following steps: S1, System Initialization and Network Partitioning: A basic network for object detection is built on a cloud server, which is divided into a shallow feature extraction network and a deep detection network. The shallow feature extraction network is deployed on multiple edge servers for local feature extraction of original images or video frames. The deep detection network is deployed on a cloud server for object detection inference and training updates based on intermediate features from the edges. S2, Fixed Class Prototype Construction and Distribution: On the cloud server, feature representations of each category are extracted and inter-category similarity is calculated based on historical datasets, public datasets, or pre-trained encoders. In a unified feature space, a fixed class prototype vector is assigned to each category, and constraints can be imposed on the relative positions of class prototypes based on similarity. All class prototype vectors are stored on the cloud server and distributed to each edge server for use in the training and inference phases. S3, Edge-Cloud Feature Collaboration and Prototype Alignment Training: Each edge server executes a shallow feature extraction network locally, performs forward computation on the locally collected data to obtain intermediate feature tensors, and uploads the intermediate features to the cloud server after compression or quantization while meeting bandwidth constraints. The cloud server uses a deep detection network to further encode and decode the intermediate features to obtain the deep feature representation of the sample. Based on the sample's class label, it selects the corresponding class prototype from a fixed set of class prototypes, constructs the prototype alignment loss and the detection task loss, and updates the parameters of the shallow feature extraction network and the deep detection network without frequently updating the weight matrix of the large-scale classifier. S4, Reflection Mechanism: After each or several training rounds, the cloud server statistically analyzes the features uploaded from different edge servers, obtaining feature distribution statistics indexed by edge server and category. These statistics include feature mean, variance, and average distance to the class prototype, and are compared with the statistics from previous training rounds. When a drastic change in the feature distribution of a certain category or the corresponding category on a certain edge server is detected, a feature stability loss is constructed to shrink the feature distribution of the current round towards historical statistical results or the class prototype. Simultaneously, samples with large errors are grouped into a reflection sample set, and the sampling weight of the reflection sample set is increased or the number of training iterations is repeated in subsequent training to achieve targeted correction of the model's weaknesses. S5, Peer Review Mechanism: The cloud server collects partial samples or intermediate feature-label pairs from each edge server, constructs a mutual evaluation data subset, and distributes these subsets to other edge servers. Each edge server uses its local current model to infer on the mutual evaluation data subset, calculates precision, recall, average precision, or class prototype-based discriminant metrics, forms mutual evaluation results for other edge models, and feeds them back to the cloud server. The cloud server aggregates these results to obtain a mutual evaluation matrix, and comprehensively evaluates the model generalization ability and data quality of each edge server. S6, Model Adaptive Improvement and Global Dynamic Fusion: Based on the peer review results and feedback from the reflection mechanism, the cloud server generates personalized training configurations for each edge server, including learning rate, regularization coefficient, class loss weights, and training rounds, and distributes them to the corresponding edge servers for the next round of local training. Simultaneously, based on the peer review score, data volume and data diversity indicators, and feature stability indicators, the cloud server assigns fusion weights to each edge model, performs weighted fusion of the model parameters or feature extraction network parameters uploaded by each edge server, obtains an updated global model, and distributes this global model to each edge server as the starting point for the next round of training or inference. Steps S3 to S6 are repeated until convergence.

[0008] The beneficial effects of this invention are as follows: (1) This invention pre-builds and distributes fixed class prototype vectors in the cloud, replacing "continuously updating the weights of large-scale classifiers" with "feature alignment to fixed prototypes". This significantly reduces the number of trainable parameters and update frequency on the edge server side, reduces the computation and storage burden on the edge, and is more suitable for resource-constrained edge-cloud collaborative scenarios. (2) The reflection mechanism proposed in this invention utilizes feature distribution statistics across training rounds and a difficult example replay strategy to explicitly constrain the features of new and error-prone categories, thereby alleviating the feature oscillation problem caused by few sample data and improving the stability and accuracy of new category detection results. (3) The mutual evaluation mechanism proposed in this invention allows different edge servers to evaluate each other's model performance under the scheduling of cloud servers. The cloud server combines the mutual evaluation results with data quality indicators to assign fusion weights to each edge model, replacing the traditional simple average global model aggregation method. This makes the global model pay more attention to edge nodes with strong generalization ability, high data quality and stable features, thereby improving the overall detection performance on multi-source heterogeneous data. (4) Through the overall design of “fixed class prototype + reflection mechanism + mutual evaluation - dynamic fusion”, this invention enables the few-sample target detection model to achieve efficient and stable joint training under the edge-cloud collaborative architecture, providing a new solution with scalability and practical value for edge intelligent perception and distributed target detection. Attached Figure Description

[0009] Figure 1 This is a flowchart illustrating the overall process of the method of the present invention.

[0010] Figure 2 This is a schematic diagram of the edge-cloud collaborative network structure of the present invention;

[0011] Figure 3 This is a schematic diagram of the mutual evaluation mechanism of the present invention;

[0012] Figure 4 This is a schematic diagram of the reflection mechanism of the present invention. Detailed Implementation

[0013] To make the objectives, technical solutions, and beneficial effects of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the following embodiments are for illustrative purposes only and do not constitute a limitation on the scope of protection of this invention. Example 1

[0014] like Figure 1 As shown, this embodiment provides a few-shot target detection edge-cloud collaborative training method based on fixed class prototypes and mutual evaluation-reflection mechanism. Its overall process includes: system initialization and network partitioning, fixed class prototype construction, edge-cloud feature collaboration and prototype alignment training, reflection mechanism, mutual evaluation mechanism, and model adaptive improvement and global dynamic fusion.

[0015] (I) System Initialization and Network Partitioning A basic object detection network based on a convolutional neural network or Transformer structure is constructed on a cloud server, and it is divided into two parts: a shallow feature extraction network and a deep detection network. The shallow feature extraction network generally includes several pre-convolutional layers or the first few layers of the backbone network, which are used to extract low-level or mid-level features; the deep detection network includes the remaining backbone layers, feature pyramids, and classification and regression detection heads, which are used to extract high-level semantic features and output the target category and location. The shallow feature extraction network is deployed on multiple edge servers, enabling the edge servers to perform preliminary feature extraction on images captured by cameras or sensors locally; the deep detection network is deployed on cloud servers, enabling high-complexity object detection inference and training updates to be performed in the cloud.

[0016] (II) Construction and Distribution of Fixed Class Prototypes: Historical or public datasets are selected, and a certain number of samples are collected for each category. Feature vectors are extracted using a pre-trained visual encoding network. The mean of the feature vectors for each category is taken to obtain the initial category vectors, and all category vectors are normalized to form an initial class prototype set. Based on the semantic relationships or confusion matrix between categories, the similarity between categories is calculated, and regularization constraints are applied to ensure that similar categories are closer together and dissimilar categories are farther apart in the feature space. The final fixed class prototype set is stored on a cloud server and distributed to various edge servers for use in calculating prototype alignment loss and mutual evaluation metrics. During training, the class prototype vectors remain fixed or are only slightly updated when preset conditions are met to ensure the stability of the learning objective.

[0017] (III) Edge-Cloud Feature Collaboration and Prototype Alignment Training During the training phase, each edge server samples image or video frames from its local data source and uses a locally deployed shallow feature extraction network to perform forward operations on the raw data to obtain intermediate feature tensors. To reduce transmission overhead, the intermediate features can be reduced in dimensionality, quantized, or made sparse before being uploaded to the cloud server via a secure communication channel. After receiving intermediate features, the cloud server inputs them into the deep detection network to obtain fused feature representations, and outputs class predictions and bounding box regression results through the detection head. For training samples with ground truth annotations, the cloud server selects class prototype vectors corresponding to the corresponding class from a fixed set of class prototypes, constructs a prototype alignment loss function, and makes the sample features align with their corresponding class prototypes. At the same time, it combines the bounding box regression loss with the traditional detection classification loss to form a joint training objective, and updates the parameters of the shallow feature extraction network and the deep detection network through backpropagation.

[0018] (iv) Reflection Mechanism After each training round, the cloud server statistically analyzes the features and labels uploaded by each edge server, calculates the feature mean, variance, and average distance between the feature and the class prototype for each edge-class combination, and compares these statistics with those from the previous round or several rounds ago. When the statistical difference between a certain class or a certain edge-class combination in adjacent rounds exceeds a preset threshold, its feature distribution is deemed unstable. In the next round of training, the cloud server adds a feature stability regularization term to the loss function, causing the feature statistics of the current round to shrink towards historical statistical results or class prototypes. In addition, samples with large losses in the previous round are automatically added to the reflection sample set, and the sampling ratio of the reflection sample set is increased, so that the model focuses on the samples that performed poorly in the previous round, realizing a self-correction process similar to a "mistake notebook".

[0019] (V) Peer Evaluation Mechanism like Figure 3 As shown, to achieve collaborative learning among edge servers, the cloud server collects a small number of anonymized samples or feature-label pairs from each edge server to form a mutual evaluation data subset. The cloud server distributes the mutual evaluation data subset to other edge servers, and each edge server uses its local current model to perform inference on the mutual evaluation data subset, calculating metrics such as accuracy, recall, mean precision, and discrimination margin based on class prototype distance, to form a mutual evaluation result for the evaluated edge model. Each edge server feeds back the mutual evaluation results to the cloud server, which then aggregates them to obtain a mutual evaluation matrix, quantitatively evaluating the generalization performance of each edge model on other edge data. By combining the local validation set performance and data quality information of each edge server, the cloud server can more comprehensively assess the contribution of each edge model.

[0020] (vi) Model Adaptive Improvement and Global Dynamic Fusion like Figure 2 As shown, the cloud server generates personalized training configurations for each edge server based on the information obtained from the mutual evaluation matrix and the reflection mechanism. These configurations include learning rate adjustment strategies, regularization coefficients, loss weights for different categories, and training epochs. This information is then distributed to each edge server to guide the next round of training. For edge servers that perform well in mutual evaluation but whose features are not stable enough, the learning rate can be appropriately reduced and the regularization strength increased. For edge servers with high data quality but low mutual evaluation scores, the weight of their new class samples in the loss can be appropriately increased. Simultaneously, the cloud server calculates fusion weights for each edge model based on peer review scores, data volume and diversity indicators, and feature stability indicators. It then performs weighted fusion of model parameters or feature extraction network parameters uploaded from each edge server to obtain a global model. The cloud server distributes the global model to each edge server for the next round of training or online inference. Through multiple rounds of iterative execution of the closed-loop process of "prototype alignment training - reflection - peer review - dynamic fusion," the method of this invention can gradually improve the overall performance and generalization ability of the few-shot object detection model.

[0021] Those skilled in the art will understand that, without departing from the spirit and essence of the present invention, equivalent substitutions or adjustments can be made to the step sequence, network structure, loss form, etc. of the above embodiments, and all such substitutions should fall within the protection scope of the present invention.

Claims

1. A collaborative training method for edge clouds in few-shot target detection based on fixed class prototypes and a mutual evaluation-reflection mechanism, characterized in that, Includes the following steps: S1, System Initialization and Network Partitioning: A basic network for object detection is built on a cloud server, and the network is divided into a shallow feature extraction network and a deep detection network. The shallow feature extraction network is deployed on multiple edge servers, and the deep detection network is deployed on the cloud server. S2, Fixed class prototype construction: On the cloud server, based on the class-level features extracted from historical datasets or pre-trained models, the similarity matrix between each class is calculated. A fixed class prototype vector is assigned to each class in a unified feature space, and the fixed class prototype vector is distributed to each edge server. During the training process, the class prototype vector is kept fixed or only limited in magnitude is allowed to be updated. S3, Edge-Cloud Feature Collaboration and Prototype Alignment Training: Each edge server uses the shallow feature extraction network to extract features from local image or video data, obtains intermediate features, and uploads them to the cloud server; the cloud server uses the deep detection network to perform forward propagation on the intermediate features to obtain sample feature representations. For samples with category labels, a prototype alignment loss is constructed based on the fixed class prototype vector of the corresponding category, so that the sample features move closer to the corresponding fixed class prototype. In this way, the training of the few-sample target detection model is completed by updating the parameters of the shallow feature extraction network and the deep detection network, without frequently updating the weight matrix of the large-scale classifier. S4, Reflection Mechanism: After each training round, the cloud server performs statistical analysis on the sample features from each edge server to obtain the distribution information such as the mean and variance of the features of each edge server and each category, and retains the historical statistical results of several rounds. In subsequent training rounds, a feature stability loss is constructed to ensure that the feature statistics of the current round are consistent with the historical statistics. A reflective sample set is constructed based on the samples with high loss, and the reflective sample set is trained first to improve the feature stability and recognition accuracy of new categories and difficult examples. S5, Peer Review Mechanism: The cloud server samples some data or feature-label pairs from each edge server to construct a mutual evaluation data subset, and then distributes the mutual evaluation data subset to other edge servers; Each edge server uses its local model to perform inference on a subset of the peer-reviewed data, calculates peer-review results such as accuracy, recall, mean precision, or discriminant metrics based on prototype distance, and feeds the peer-review results back to the cloud server. The cloud server then aggregates and constructs a peer-review matrix and evaluates the generalization ability and data quality of each edge model. S6, Model Adaptive Improvement and Global Dynamic Fusion: Based on the peer review results and reflection statistics, the cloud server generates personalized training configurations for each edge server, including learning rate, regularization strength, and loss weights for each category, and distributes them to the corresponding edge server for the next round of local training. At the same time, the cloud server assigns fusion weights to each edge model based on peer review scores, data quality scores, and feature stability indicators, performs weighted fusion of the parameters of each edge model to obtain a global model, and distributes the global model parameters to each edge server as the initial model for the next round of training or inference. Repeat steps S3 to S6 until the preset convergence condition is met to obtain the final global model for few-sample target detection.

2. The method according to claim 1, characterized in that, In step S2, the construction of the fixed class prototype vector includes: using a pre-trained visual encoding network to extract features from samples of each category and calculating the feature center of each category as the initial vector of the category; or using a pre-trained text encoding model to encode the semantic labels of the categories into semantic vectors; and normalizing and constraining each category vector in a unified feature space according to the similarity matrix, so that the class prototype vectors of similar categories are closer in the feature space and the dissimilar categories are farther apart.

3. The method according to claim 1, characterized in that, In step S3, the prototype alignment loss includes at least one of the following: (1) Constructing a minimum distance loss based on the Euclidean distance between sample features and corresponding class prototype vectors; (2) A softmax classification loss constructed with negative distance or similarity as input is used to replace the weight matrix of the traditional linear classifier to achieve class discrimination; (3) A joint loss function that includes both bounding box localization loss and class prototype-based classification loss.

4. The method according to claim 1, characterized in that, In step S4, the reflection mechanism further includes: (1) When the difference in the mean or the increase in the variance of a new category in adjacent training rounds exceeds a preset threshold, the weight of the category in the loss function is automatically increased, or the learning rate of the corresponding sample is reduced, in order to suppress the drastic shift in the feature distribution. (2) The samples with larger errors in the previous round are used to form a set of difficult examples. In the next round of training, the sampling ratio or the number of repetitions of the set of difficult examples is increased to achieve targeted correction of the incorrect predictions.

5. The method according to claim 1, characterized in that, The mutual evaluation mechanism in step S5 processes the mutual evaluation data subset in an anonymous or encrypted manner. The edge server only receives de-identified data or feature-label pairs, thereby achieving performance evaluation of other edge models without directly sharing the original privacy data.

6. The method according to claim 1, characterized in that, In step S6, the fusion weights of each edge model are calculated based on a weighted combination of the following metrics: (1) Average performance index across the edge obtained from mutual evaluation; (2) The data volume and category coverage of the corresponding edge server; (3) Characteristic stability indicators of the reflection mechanism assessment; The cloud server performs a weighted average or a graph-based weighted aggregation on the parameters of each edge model based on the fusion weights to obtain a global model.

7. A few-sample target detection edge cloud collaborative system for implementing the method of any one of claims 1-6, characterized in that, include: Multiple edge servers are used to deploy shallow feature extraction networks and perform feature extraction and local training on local data; A cloud server is used to deploy a deep detection network and a global control module. The global control module includes a fixed class prototype construction submodule, a reflection mechanism submodule, a mutual evaluation task scheduling and aggregation submodule, and a global dynamic fusion submodule, for executing the cloud-related steps in the method of any one of claims 1-6; and a communication network for connecting the cloud server and each edge server.

Images