Cross-terminal collaborative training method for small sample model based on federated learning

Abstract

The application discloses a small sample model cross-terminal cooperative training method based on federal learning, relates to the technical field of small sample learning, and is particularly suitable for application scenarios with significant data heterogeneity of smart home devices, and comprises the following steps: a terminal device constructs small sample device data locally, generates device data abstract containing class coverage, intra-class dispersion, inter-class separability, signal quality, feedback confidence and gradient statistics, and uploads the device data abstract to a federal server; the device data abstract is used to calculate the device data difficulty index and conflict prior index of each terminal, the device data difficulty is normalized, and a unified sampling constraint is issued; gradient conflict analysis is performed on model updating, adaptive weighted aggregation or clustering aggregation is realized in combination with a cross-group interference matrix; the in-group cooperation degree and global verification fluctuation degree are continuously monitored, and a strategy is dynamically adjusted until the model converges stably, so that the problems of data heterogeneity and small samples of traditional federal learning in smart home devices are solved, and the cooperative training effect of the model among different devices is improved.

Description

Technical Field

[0001] This invention relates to the field of few-shot learning technology, and more specifically, to a method for cross-terminal collaborative training of few-shot models based on federated learning. Background Technology

[0002] Federated learning, as a distributed machine learning method, allows multiple terminal devices to jointly train a global model without sharing local data, and has been widely used in fields such as smart homes and health monitoring. However, in smart home devices, data heterogeneity and the small sample size problem remain major challenges for current technology. Different smart devices have significant differences in the types, frequency, and quality of data collected, which leads to conflicts in model updates between devices, affecting the training efficiency and performance of the global model. Existing federated learning methods primarily rely on a unified training strategy and weighted aggregation. However, in scenarios with highly heterogeneous data and small data volumes, such as smart home devices, traditional training methods struggle to effectively address the differences in data difficulty and data conflicts between devices. This results in slow global model convergence, unstable performance, and poor generalization ability. Furthermore, traditional methods often require uploading large amounts of raw data, potentially posing a privacy risk and limiting their widespread adoption and use in practical applications. Therefore, there is an urgent need for a new type of federated learning method that can effectively handle the problems of data heterogeneity and small sample size in smart home devices, and improve the effect of collaborative training and the performance of the global model while ensuring data privacy.

[0003] To address the above problems, this invention proposes a solution. Summary of the Invention

[0004] To overcome the aforementioned deficiencies of the prior art, embodiments of the present invention provide a method for cross-terminal collaborative training of few-shot models based on federated learning, in order to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides the following technical solution: A method for cross-terminal collaborative training of few-shot models based on federated learning, including the following steps; Step S1: Collect interactive data fragments and context information of smart home devices, perform alignment, normalization and desensitization processing on the data of different devices, encode the samples into vector representation through the edge embedding model, calculate the class centroid, intra-class dispersion and inter-class separability of the data of each device, and statistically analyze the signal quality mean vector, feedback confidence mean, gradient mean vector and gradient norm mean and variance, generate device data summary and upload it to the federated server; Step S2: The federated server normalizes the device data summary, calculates the device data difficulty index and conflict prior index. The device data difficulty index is calculated by intra-class dispersion and inter-class separability. The conflict prior index is calculated by the similarity and deviation angle of the terminal update direction relative to the reference direction. The server sets the boundary of the collaborative group, conflict threshold, interference threshold, consistency threshold and volatility threshold, compares the index with the threshold to determine the heterogeneous collaborative group identifier, and sends the device data construction configuration to constrain terminal sampling. Step S3: Calculate the cluster optimization baseline and conflict score within the group. If the conflict score exceeds the conflict threshold, perform a delay or shadow verification on the update. Calculate the cross-cooperative group intersection interference matrix and compare it with the interference threshold to determine isolation aggregation or cluster aggregation. Generate the device contribution weight and update the model based on the conflict score, device data difficulty index, signal quality mean vector and feedback confidence mean. Step S4: At the end of each round, calculate the intra-group synergy score and verify the volatility. If the consistency score is not less than the consistency threshold and the volatility is not greater than the volatility threshold, proceed to the next round; otherwise, return to step S2 to adjust the configuration or end the iteration, and output the model parameters and aggregated explanation log.

[0006] In a preferred embodiment, step S1 includes the following: The terminal constructs small sample learning device data, which includes: preprocessing, labeling and augmenting local data, and dividing it into support set and query set; The generated device data summary includes: calculating and recording the category coverage of the device data, the intra-class dispersion and inter-class separability of each category, the signal quality vector, the feedback confidence, and the gradient statistics during the local few-sample training process; Intra-class dispersion is obtained by calculating the average distance from the vector representation of all supporting samples of the same class to the centroid of that class; Inter-class separability is obtained by calculating the distance between the centroids of different classes; The device data summary must include at least the category coverage vector, intra-class dispersion vector, inter-class separability matrix, signal quality mean vector, feedback confidence mean, gradient norm mean and variance, and be reported through an encrypted channel; Before generating a device data summary, the terminal will perform multiple rounds of small-sample training locally to statistically analyze the gradient norm and its fluctuations during the training process. The signal quality vector includes one or more of the following: speech signal-to-noise ratio, data missing rate, sensor drift intensity, and network latency fluctuation. Feedback confidence is determined by explicit user feedback, implicit feedback, and cross-session consistency.

[0007] In a preferred embodiment, step S2 includes the following: When calculating the difficulty index of device data, the federal server comprehensively considers the overall level of intra-class dispersion, the overall level of inter-class separability, and the gradient variance reported by each terminal. When calculating the conflict prior index, the deviation between the terminal update gradient direction and the global average direction, the mean gradient norm, and the feedback confidence level are comprehensively considered. Based on the calculated device data difficulty index and conflict prior index, the federated server divides the terminals into different cooperative groups and generates targeted device data sampling constraints for each cooperative group. The sampling constraints include one or more of the following: support set size, sampling weights for each class of samples, and low-quality sample suppression coefficient, which are used to guide the terminal to adjust the local device data construction in the next round of training.

[0008] In a preferred embodiment, step S3 includes the following: Gradient direction conflict analysis includes: calculating a cluster optimization baseline for each heterogeneous collaborative group; For each terminal update within the collaboration group, calculate the cosine of the direction angle between the update and the cluster optimization baseline, and calculate the conflict score based on the cosine value and the relative magnitude of the update within the collaboration group. The adaptive weighted aggregation strategy is as follows: based on the conflict score of the terminal, the device data difficulty index, and the conflict prior index, a device contribution weight is assigned to each terminal update, and the weight is negatively correlated with the conflict score. The clustering and aggregation strategy is as follows: when a serious update direction conflict is identified between different heterogeneous collaborative groups, the terminals in the conflicting collaborative groups are re-clustered, and independent models are generated for each cluster.

[0009] In a preferred embodiment, step S4 includes the following: The intra-group collaboration score is calculated based on the average similarity between the update directions of all terminals within the collaboration group; The calculation of global model performance volatility is based on the degree of dispersion of the performance metrics exhibited by the model on a set of independent validation device datasets. The dual criteria are a preset consistency threshold and a performance volatility threshold; The rules for dynamic decision-making based on dual criteria include: if the consistency score is higher than its threshold and the performance volatility is lower than its threshold, then the current strategy is maintained and training continues. If the consistency score is lower than its threshold, a return to step S2 is triggered to adjust the device data sampling constraints or re-divide the collaboration groups; If the performance volatility exceeds its threshold, return steps S2 and S3 are triggered to strengthen conflict suppression or enable cluster aggregation.

[0010] The technical effects and advantages of this invention's method for cross-terminal collaborative training of few-shot models based on federated learning are as follows: This invention effectively solves the problems of training instability and poor performance caused by data heterogeneity and small sample size in smart home devices by introducing a device data difficulty normalization and conflict prior analysis mechanism. Specifically, this invention calculates the device data difficulty index and conflict prior index by calculating the device data summary of each terminal device, thereby normalizing the device data difficulty between different devices and avoiding training target deviation caused by data heterogeneity. An adaptive weighted aggregation and clustering aggregation strategy combining gradient conflict analysis and cross-group interference matrix was adopted. When there is a large conflict in model updates between devices, it can effectively alleviate the conflict and improve the stability of the training process. By continuously monitoring the intra-group synergy and global validation volatility, the strategy is dynamically adjusted to ensure the collaborative training effect of the global model across different devices. This method not only improves the convergence and generalization capabilities of federated learning in smart home devices under the challenges of small sample sizes and data heterogeneity, but also significantly improves the training stability and effectiveness of the global model while ensuring data privacy, and has broad application prospects. Attached Figure Description

[0011] Figure 1 This is a schematic diagram of the cross-terminal collaborative training method for few-shot models based on federated learning, as presented in this invention. Detailed Implementation

[0012] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0013] Example Please see Figure 1 As shown, this invention discloses a method for cross-terminal collaborative training of few-shot models based on federated learning, including the following steps: Step S1: Collect interactive data fragments and context information of smart home devices, perform alignment, normalization and desensitization processing on the data of different devices, encode the samples into vector representation through the edge embedding model, calculate the class centroid, intra-class dispersion and inter-class separability of the data of each device, and statistically analyze the signal quality mean vector, feedback confidence mean, gradient mean vector and gradient norm mean and variance, generate device data summary and upload it to the federated server; Step S2: The federated server normalizes the device data summary, calculates the device data difficulty index and conflict prior index. The device data difficulty index is calculated by intra-class dispersion and inter-class separability. The conflict prior index is calculated by the similarity and deviation angle between the terminal update direction and the reference direction. The server sets the boundary of the collaborative group, conflict threshold, interference threshold, consistency threshold and volatility threshold, compares the index with the threshold to determine the heterogeneous collaborative group identifier, and sends the device data construction configuration to constrain terminal sampling. Step S3: Calculate the cluster optimization baseline and conflict score within the group. If the conflict score exceeds the conflict threshold, perform a delay or shadow verification on the update. Calculate the cross-cooperative group intersection interference matrix and compare it with the interference threshold to determine isolation aggregation or cluster aggregation. Generate the device contribution weight and update the model based on the conflict score, device data difficulty index, signal quality mean vector and feedback confidence mean. Step S4: At the end of each round, calculate the intra-group synergy score and verify the volatility. If the consistency score is not less than the consistency threshold and the volatility is not greater than the volatility threshold, proceed to the next round; otherwise, return to step S2 to adjust the configuration or end the iteration, and output the model parameters and aggregated explanation log.

[0014] In step S1, interactive data fragments and contextual information from smart home devices are collected. Data from different devices is aligned, normalized, and anonymized. Samples are encoded into vector representations using an edge-side embedding model. The class centroid, intra-class dispersion, and inter-class separability of each device's data are calculated. The mean vector of signal quality, mean feedback confidence, average gradient vector, and mean and variance of the gradient norm are also statistically analyzed. A device data summary is generated and uploaded to the federated server. Specific content includes: Collect interactive data fragments and contextual information from smart home devices, including but not limited to smart speakers, sensors, and appliance controllers. Their local data exhibits significant heterogeneity in terms of type, frequency, and quality. The collected data segments specifically include voice commands, environmental monitoring readings, equipment status logs, and user interface interaction sequences; Contextual information includes the network conditions where the device is located, its approximate spatial location, the time of use, and the device's own load status; Time alignment, unit normalization, and desensitization of privacy-sensitive fields are performed on the above multi-source heterogeneous data to form the basis for subsequent device data construction. Based on the meta-learning paradigm, the terminal divides the support set and query set into semantic categories from the local heterogeneous data stream. For categories with scarce samples, data augmentation technology limited to the local device is used to expand them. Each terminal constructs a small sample of learning device data locally according to predetermined rules, generates a corresponding statistical summary of device data, and reports it to the federated server. The core objective is to collect quantitative indicators from each terminal that can reflect the difficulty and distribution differences of device data, which will be used for subsequent difficulty normalization and conflict analysis. Each terminal obtains data fragments related to personalized services from local logs, sensor acquisition modules, or interactive service modules. The data fragments include at least one or more of the following: voice interaction fragments, touch button operation sequences, in-application behavior sequences, and device context information. The device context information includes network status, coarse-grained location area identifier, time period identifier, motion status, device power and temperature status, etc. After collection, the data is processed by timestamp alignment, sampling rate normalization, missing segment annotation, outlier segment removal, and numerical normalization. For voice data, edge noise reduction and silence segment clipping are performed, and for touch sensor data, jitter smoothing and outlier filtering are performed. The preprocessing stage also desensitizes or removes identifiable fields, including contact identifiers, precise locations, reversible identity markers, and fields containing original text content or reproducible content; only statistical features or embedding representations required for training are retained. After preprocessing, samples are labeled with multidimensional tags, which include at least: service intent category or function category, interaction scenario category, interaction result category, content preference category, and device status category. The tags are jointly confirmed by the terminal-side rule engine, the terminal-side lightweight model, or explicit user feedback. Samples with inconsistent tags are marked as low-confidence samples. The terminal divides local data into a support set and a query set according to the meta-learning or few-shot learning framework. The support set is used to update the local model parameters, and the query set is used to evaluate the adaptability of the updated support set. When constructing the data, it is required to cover as many categories as possible that appear locally. For example, different service intent categories or different interaction scenario categories are regarded as different device data categories. If the number of samples in some categories is too small, data augmentation methods that can be implemented on the device side are used to increase the number of samples in the minority class, including time jitter, fragment clipping and splicing, noise perturbation, feature space mixing, or on-device synthesis of generative models. The terminal records the number of samples of each category in the device data constructed in this round in the support set and query set as input for category coverage. To quantify the internal and external differences in device data, each terminal trains an initial embedding model locally to map speech, touch sequences, behavioral sequences, or multi-sensor sequences into a unified vector representation. The embedding model can employ a temporal convolutional network, temporal Transformer, dual-tower metric learning network, or prototype network encoder to encode each sample as a vector representation, denoted as the... The set of sample vectors for each category in the support set is: Its centroid is defined as: ;in, For the first The class supports a certain number of samples; It should be noted that the centroid is the average value of the vector representations of the supporting samples of the same category. That is, it is obtained by summing the vector representations of all supporting samples of the category and then dividing by the number of supporting samples of the category. The sample vector representation is obtained by encoding speech segments, touch sequences, behavior sequences or multi-sensor sequences by the on-device embedding model. The vector dimension is predetermined by the embedding model structure and fixed on the terminal side. The edge-embedded model is a pre-trained lightweight neural network, such as a variant of MobileNetV3 or TinyBERT, whose output is a fixed-dimensional feature vector. The model has been initially trained on a public dataset before participating in federated learning, and some parameters of its feature extractor are fixed on the edge. The intra-class scatter can be calculated based on the centroid, and the formula can be expressed as: And calculate the inter-class separability, the formula can be expressed as: ; Intra-class dispersion characterizes the degree to which sample vectors of the same class are dispersed around the centroid of that class, and can be obtained by averaging the distances of all support sample vectors of that class to the centroid. Inter-class separability characterizes the degree of distinction between different classes, and can be calculated by the distance between the centroids of different classes. The distance is calculated using either Euclidean distance or cosine distance, and one distance method is always used in the same implementation process and does not change with each round. By comparing the average separability between different categories with the intra-class dispersion, we can initially measure the difficulty of device data. Larger and Smaller values ​​indicate that the categories are easy to distinguish, while larger values ​​indicate that the equipment data is more difficult to analyze. The terms "larger" and "smaller" are determined using a threshold criterion: the server calculates the mean and standard deviation of the intra-class dispersion and inter-class separability statistics reported by all terminals during the same training phase. A "larger" threshold is defined as an inter-class separability higher than its mean plus standard deviation, while a "smaller" threshold is defined as an intra-class dispersion lower than its mean minus standard deviation. All other cases are defined as "moderate." These thresholds are updated at the beginning of each training phase and distributed to all terminals as consistency criteria. To address the issue of significant differences in data quality at the endpoints, a signal quality vector is introduced. The system includes metrics such as voice signal-to-noise ratio, frame drop rate, sampling completeness, touch trajectory stability, sensor drift intensity, network latency fluctuation, and edge resource usage fluctuation. These metrics are used to screen low-quality samples and record feedback confidence. Feedback confidence is determined by explicit user feedback, implicit feedback, and cross-session consistency. The calculation of feedback confidence uses a consistency scoring method: the consistency of feedback in adjacent sessions for the same service intent or the same interaction result is scored, and points are deducted based on whether explicit and implicit feedback contradict each other. When the consistency score is lower than a preset threshold, the sample or label is marked as low confidence, and the corresponding low confidence ratio is reported synchronously in the device data summary. When explicit and implicit feedback contradict each other or cross-session consistency is insufficient, the corresponding sample or label is marked as low confidence, thus providing a basis for subsequent weight adjustments. For example, for smart speakers, the signal quality vector mainly includes the voice signal-to-noise ratio and wake word detection confidence; for temperature and humidity sensors, it mainly includes the frequency of reading jumps and self-calibration history error; for network cameras, it mainly includes image blur index and key frame transmission success rate. These indicators directly reflect the differences in data reliability caused by different devices due to their type, deployment environment and hardware status. It should be noted that the signal quality vector is generated locally by the terminal-side acquisition and preprocessing module, and each dimension is a reproducible numerical indicator: the speech signal-to-noise ratio is calculated by the ratio of effective speech energy to background noise energy; the frame drop rate and missing rate are calculated by the ratio of the actual number of data segments arriving within the continuous sampling window to the theoretical number of data segments that should arrive; the sampling integrity is calculated by the ratio of the effective sampling length to the predetermined sampling length; the touch trajectory stability is calculated by the degree of velocity or acceleration fluctuation of the trajectory point sequence; the sensor drift intensity is calculated by the zero-point offset statistics output by the sensor stationary segment; and the network latency fluctuation is calculated by the degree of dispersion of communication latency in multiple consecutive uploads. The terminal performs several rounds of few-sample meta-training locally, for example using a prototype network or MAML, and records the updated gradient norm in each round. and direction, by means of the gradient norm and variance Statistical analysis can reflect the smoothness and difficulty of device data. For example, if a terminal has a large gradient norm and high variance in most rounds, it may indicate that its device data design has overlapping category boundaries, too many noisy samples, or situational distribution drift, making it difficult to achieve stable convergence. Taking the prototype network as an example, its local training process is as follows: for support set samples, calculate the prototype vector of its class; for query set samples, calculate the distance between them and the prototype vectors of each class, and use the cross-entropy loss function to optimize the embedding model parameters. Device data summary packaging and uploading: The terminal packages the above statistics into device data summary records, specifically including: category coverage vector. ,in To support the collection of the first Sample percentage of each class, within-class scatter vector Inter-class separability matrix Signal quality mean vector Mean of feedback confidence Gradient norm mean With variance In addition, terminal identification information, device data number for this round, and device computing power information are also included. Each terminal reports its data summary to the federated server in an encrypted channel. The federated server receives statistical information from the terminal without obtaining the original interaction data and tags, providing a basis for subsequent difficulty normalization and conflict analysis. Each dimension of the category coverage vector corresponds to the percentage of samples of a category in the support set; each dimension of the intra-class dispersion vector corresponds to the intra-class dispersion value of a category; each element of the inter-class separability matrix corresponds to the distance between a pair of category centroids; the signal quality mean vector is the average value of each dimension of the signal quality index on the current round of device data samples; the feedback confidence mean is the average confidence value of all samples in the current round of device data; the gradient norm mean and gradient norm variance are obtained by the terminal performing local meta-training and statistically analyzing the gradient norm of each round of parameter updates; the terminal uploads only the above statistics and does not include the original interactive content that can be reversibly restored or the identifiable identity fields. Obtain multi-dimensional device data summaries from different terminals. These summaries fully describe the internal structure, quality, and training dynamics of the data construction for each terminal device, laying the foundation for evaluating the differences between device data and potential aggregation conflicts. They involve multiple parameters such as quality vectors, device data features, and gradient statistics. The terminology definitions must be consistent, and subsequent steps will directly refer to these parameters as the basis for decision-making.

[0015] In step S2, the federated server normalizes the device data summary, calculates the device data difficulty index and conflict prior index. The device data difficulty index is calculated from the intra-class dispersion and inter-class separability, and the conflict prior index is calculated from the similarity and deviation angle between the terminal update direction and the reference direction. The server sets the boundary of the cooperative group, conflict threshold, interference threshold, consistency threshold, and volatility threshold. The index is compared with the threshold to determine the heterogeneous cooperative group identifier, and the device data construction configuration constrains the terminal sampling. The specific content includes: Normalization analysis and conflict prior construction are performed on the device data summaries from each terminal on the federated server side to alleviate the training target bias caused by the large difference in difficulty of small sample device data constructed by different terminals. The device data difficulty index TDI and conflict prior index CPI of each terminal are calculated based on the statistics in step S1, and a unified device data sampling constraint is generated accordingly to guide the terminal to construct more consistent device data in subsequent rounds. The server dynamically calculates and updates the global mean and standard deviation used for exponential normalization based on the statistics reported by all terminals. Weighting coefficients in the Equipment Data Difficulty Index (TDI) And the weighting coefficients in the conflict prior index CPI Its initial value is preset based on the historical training logs of smart home scenarios, and fine-tuned according to the convergence monitoring results during the training process; The boundaries of the collaborative groups are dynamically determined based on the distribution of TDI and CPI values ​​of all terminals in the current round, through cluster analysis or quantile partitioning. The federated server first normalizes the statistics for each terminal. Let the intra-class scatter vector of a terminal be... The inter-class separability matrix is To simplify the calculation, the average separability can be obtained by averaging the elements in the inter-class separability matrix. The equipment data difficulty index is defined according to the following formula: ;in, For the number of categories, The weighting coefficient can be set based on experience or learned through cross-validation. This index takes into account the impact of intra-class dispersion, inter-class separability, and gradient variance on the difficulty of device data. The larger the value, the more difficult the device data is. The device data difficulty index is determined by three factors: the first is the overall level of intra-class dispersion, which reflects whether samples of the same class are scattered; the second is the overall level of inter-class separability, which reflects whether different categories can be distinguished; and the third is the fluctuation of the local training gradient, which reflects whether the training is stable. The above three statistical measures are linearly normalized according to the minimum and maximum values ​​of all terminals and then combined for calculation. The weight coefficients are set by the server based on historical training logs or the stability index of the verification device data set during the initialization phase of training, and are fine-tuned according to the convergence monitoring results in subsequent phases. In smart home scenarios, high intra-class dispersion may be due to voice commands with the same intent being issued by different family members, or the same environmental state being perceived by sensors at different locations and with different precision; low inter-class separability may be due to similar voice commands with different control intents, or overlapping state events of different devices in the feature space. The device data difficulty index quantifies the learning difficulties caused by the inherent heterogeneity of the scenario. The conflict prior index aims to measure whether the update direction of a terminal device's data deviates from the possible target direction of the entire federation. The federation server first calculates the average gradient vector of the previous round of reports from each terminal. Then, the gradient average vector for each terminal... The formula for calculating cosine similarity can be expressed as: And calculate the deviation angle. ,when When the angle is large and obtuse, it indicates that the gradient direction of the terminal conflicts with the global direction; when it is acute and small, it indicates that the directions are consistent. In cross-device collaborative training for smart homes, conflicts often arise from the different distribution of device data, such as environmental data in the bedroom and living room, leading to localization of model optimization direction. Low-quality data, such as noisy speech and drifting sensor readings, can produce misleading gradients. The Conflict Prior Index (CPI) identifies such harmful updates that may be caused by data heterogeneity and quality issues in advance by combining update direction bias, gradient magnitude, and data confidence. To ensure reproducibility of the judgment, the server uses a threshold criterion: when the gradient direction similarity is lower than the similarity threshold, it is judged as an inconsistency in direction; when the deviation angle is higher than the angle threshold, it is judged as a conflict in direction. The similarity threshold and angle threshold are obtained by the server calculating the quantiles of the similarity distribution of all terminals in the most recent rounds, and are fixed once at the beginning of each training phase. Combining the gradient norm mean and data quality, a conflict prior index can be defined: ;in, It's weight. The weighted gradient associated with terminal i, To maximize the mean of all terminal gradients for normalization, It is the mean of the confidence level of the terminal feedback. A high value indicates that the update of this terminal is expected to be inconsistent with most terminals and may cause conflicts; Difficulty-based collaborative group generation and unified sampling constraint generation, based on each terminal's and The federated server performs clustering or grouping of terminals into collaborative groups, for example, by... The grade points are defined as low, medium, and high difficulty levels. The data is divided into two levels: low-conflict and high-conflict. Then, it is combined into several collaborative groups. The difficulty distribution of the device data of the terminal in each collaborative group is relatively concentrated. For the low-difficulty and low-conflict collaborative groups, the terminal can be required to appropriately increase the proportion of difficult device data, such as increasing the proportion of long-tail intent categories, weak context conditions, or sparse scenarios. For high-difficulty or high-conflict collaborative groups, the terminal is required to reduce the difficulty in the next round of sampling. This can be achieved by reducing nearly duplicate samples, increasing the diversity of the support set, and suppressing the probability of low-quality fragments participating. At the same time, noisy samples can be reduced through edge cleaning or relabeling. The unified sampling constraints are generated by the server and distributed to the terminal. The configuration includes the support set size, the upper limit of samples for each category, the ratio of the support set to the query set, the sampling weights of difficult and easy examples, and the low-quality sample suppression coefficient. Normalization feedback and local adjustment: The terminal receives sampling constraints from the federated server and resamples and reconstructs device data on the local dataset. For example, the terminal adjusts the support set construction strategy according to the class weights issued by the server to ensure that the coverage of each class conforms to the global average. For samples with low feedback confidence or poor quality indicators, the participation probability is reduced according to the suppression coefficient given by the server. For device data that is too difficult, the difficulty can be reduced by feature perturbation, context hierarchical sampling or fragment resegmentation that can be implemented on the terminal side. Before executing the next round of local training, the terminal generates a new support set and query set based on the new sampling strategy. During training, the distribution of device data and the model state across different terminals change over time. This is especially true in consumer electronics scenarios, where user preferences, device states, and usage contexts exhibit significant non-stationarity. The federated server periodically re-evaluates the data of each terminal. and The boundaries and weight coefficients of the collaborative groups are updated. If some terminals are found to be consistently extremely difficult or continuously conflicting with the global direction, additional measures can be taken, such as adopting a local personalized strategy for their models, reducing their participation frequency, or selecting a more suitable cluster model for initialization. Without directly accessing the raw edge data, this method unifies the difficulty distribution of data construction across different terminal devices and predicts potential conflicts, laying a quantitative foundation for subsequent device contribution weight allocation and conflict resolution. All newly introduced parameters, such as... , The results, such as defining the boundaries of collaborative groups, will be output in the implementation results for adjustment during the actual implementation.

[0016] In step S3, a cluster optimization baseline is calculated within the group, and a conflict score is calculated. If the conflict score exceeds the conflict threshold, the update is deferred or shadowed. The cross-cooperative group intersection interference matrix is ​​calculated and compared with the interference threshold to determine isolation aggregation or cluster aggregation. Based on the conflict score, device data difficulty index, signal quality mean vector, and feedback confidence mean, device contribution weights are generated and the model is updated. Specific content includes: The allocation strategy for device contribution weights clearly favors terminals with high data quality and high feedback confidence, such as new smart speakers located in quiet environments. For terminals with moderate data difficulty and whose update direction aligns with the consensus within the collaboration group, the weight of terminals with high data heterogeneity and significant quality fluctuations, such as old sensors or mobile devices, will be dynamically reduced unless their update direction is verified to be beneficial to the global model. A deeper gradient conflict analysis is performed on updates uploaded from the terminal, and an adaptive aggregation strategy is formulated accordingly. The core of this approach is not only to identify potential conflicts, but also to quantitatively analyze the sources and extent of these conflicts, and to dynamically adjust the device contribution weights and model synchronization strategies based on the analysis results, thereby improving the stability and generalization of the federated model. For each heterogeneous cooperative group obtained after dividing into cooperative groups The server aggregates terminal updates from this collaboration group in this round. , This represents the amount of model parameter updates made by terminal i in this round of training. This update amount is calculated based on the gradient of the local data of terminal i and is used to adjust the model weights and calculate the cluster optimization baseline. The formula can be expressed as: , Indicates heterogeneous collaborative grouping Model update volume of all terminals The weighted average of the cluster optimization benchmarks represents the average optimization direction of the collaborative group and is an estimate of the data target for this type of device. The conflict score is composed of the degree of directional deviation and the degree of impact. The degree of directional deviation is obtained by the angle between the terminal update and the reference direction within the collaborative group. The degree of impact is determined by the update magnitude of the terminal, the difficulty of the device data, and the reliability of the data. The server sets a conflict threshold for the conflict score. When the conflict score is higher than the conflict threshold, it is judged as a strong conflict update and enters the suppression or verification process. The conflict threshold is obtained by the server calculating the quantile of the historical distribution of the conflict score and is fixed once in each training phase. Update each terminal Calculate the cosine of the angle between it and the cluster optimization baseline within the collaborative group: Define gradient conflict metric A larger value indicates a greater deviation of the update direction from the consensus within the collaborative group. To distinguish between acceptable noise and genuine conflicts, it is also necessary to combine the step size and the degree of influence to set the degree of influence. ; Final overall definition conflict score: Set conflict threshold Based on the distribution of all terminal conflict scores in historical rounds, and dynamically set according to preset quantiles, the following will be implemented. The conflict score is identified as a strong conflict, and the terminal is updated. The server starts the shadow verification mechanism: using a small portion of the retained verification set that covers multiple types of smart home device data, the impact of the update on the model performance is independently evaluated, and the update is decided to be adopted, downgraded or discarded based on the verification results. Although terminals are assigned to specific heterogeneous cooperative groups, their updates may have a positive or negative impact on the optimization direction of other cooperative groups. The server calculates the cross-interference matrix for each cooperative group against other cooperative groups. The formula can be defined as: ,like A negative value or value below the threshold indicates a collaborative group Update overall and collaborative groups If the update directions are opposite or inconsistent, there may be a conflict in device data targets. The server identifies conflicting pairs based on the cross-interference matrix and decides whether to adjust the collaborative group accordingly. and The aggregation employs an isolation strategy, such as calculating local and global models separately; Each element of the interference matrix represents the influence of the overall update direction of one heterogeneous collaborative group on the reference direction of another heterogeneous collaborative group. A negative value indicates that the directions are opposite, and a value close to zero indicates that the correlation is weak. The server sets an interference threshold. When the matrix element is lower than the interference threshold, it is determined that there is a significant target conflict between the two collaborative groups, and isolation aggregation or cluster training strategies are adopted for the two collaborative groups. Adaptive device contribution weight allocation, server based on terminal , , And the cross-group interference matrix, contributing weights to each update allocation device. The weights can be calculated using the following function: ;in, The conflict sensitivity coefficient, It is a moderating factor at the coordinating group level, used to encourage participation in low-conflict coordinating groups; The inputs for device contribution weights include at least: terminal conflict score, device data difficulty index, mean signal quality, and mean feedback confidence. The server first normalizes each input to a uniform range, then calculates the weights using a weighted or exponentially decaying method, and normalizes the weights of all terminals in the same round to ensure that their sum meets preset constraints, thereby guaranteeing the stability of the step size of the global model update. To prevent terminals from being permanently excluded due to weights remaining at zero for an extended period, the server can set a lower limit for the weights and use a shadow verification mechanism to determine whether to reinstate their participation. The final global model update is as follows: For updates identified as having strong conflicts or significant interference, such as Higher than the set threshold The server can temporarily suspend its participation in aggregation or include it in the shadow verification phase, which evaluates on a small verification set whether the update is indeed helpful to the device data target; if the performance is poor after verification, it is directly discarded or its weight is reduced. Clustering and personalized output: For situations where cross-group interference is severe and cannot be resolved by weight adjustment, the server further clusters the terminals according to device data characteristics. Each cluster trains an independent global model or a personalized initialization. The terminal can download the corresponding initialization model based on its cluster identifier and then train it locally. This ensures the consistency of data from devices within the same cluster while avoiding interference between data from devices in different clusters. Clustering can be performed using spectral clustering or based on... , , The hierarchical clustering method completes this automatically; Taking hierarchical clustering based on TDI and CPI as an example, the process is as follows: calculate a two-dimensional feature vector composed of the TDI and CPI values ​​of all terminals; use Euclidean distance as a similarity measure, use the Ward connection method to perform hierarchical clustering, and determine the final clustering result according to the preset inter-cluster distance threshold or the specified number of clusters. To improve security and interpretability, the server records the conflict score, weight, and isolation status of each update during aggregation, forming an aggregation interpretation log. If the terminal has questions about its weights or aggregation results, it can query the log for explanation. In addition, it can combine the signal quality and feedback confidence in the device data summary to provide a reasonable explanation for weight adjustments and prevent malicious updates or abnormal behavior. Based on the normalization of device data difficulty, a fine-grained analysis of the update direction and intensity of each terminal was performed to identify potential conflict sources. Strategies such as weighted aggregation and clustered aggregation were employed to prevent the negative impact of conflict updates on the global model. Key output results include gradient conflict metrics for each terminal. Equipment contribution weight Cross-group interference matrix The aggregation model parameters of each cluster are used as inputs for subsequent decisions.

[0017] In step S4, at the end of each round, the intra-group synergy score and validation volatility are calculated. If the consistency score is not less than the consistency threshold and the volatility is not greater than the volatility threshold, the next round begins; otherwise, the process returns to step S2 to adjust the configuration or end the iteration. Model parameters and aggregated explanation logs are output, including: Throughout the federated training process, the model's convergence status and stability are monitored. Based on dual criteria, it is determined whether the device data normalization strategy or aggregation strategy needs to be readjusted, thereby achieving dynamic iterative scheduling. This protects the stability of the training process while avoiding oscillations caused by over-adjustment. For each collaboration group The consistency score for updates within a collaborative group is calculated using the following formula: The score range is A larger value indicates greater consistency in update directions within the collaboration group; a consistency threshold is set. ,when If this occurs, it indicates that there are still significant directional differences within the collaborative group, and it is necessary to return to step S2 to adjust the sampling strategy or re-divide the collaborative group. The consistency score is obtained by statistically analyzing the consistency between the update direction of each terminal in the collaborative group and the reference direction of the collaborative group. The average value of the angle similarity can be used as the consistency score, so that the value range is fixed and comparable. A consistency threshold is set. When the consistency score is lower than the consistency threshold, it is determined that there is still a significant directional divergence in the collaborative group, and the process of returning to device data difficulty normalization and dividing collaborative group update is triggered. Maintain a separate validation dataset, including small sample data from various terminals and categories. Perform small sample evaluations on the validation dataset using the current global model or cluster models, and record the performance metrics for each device. Let the validation dataset contain a total of [number missing]. Data for each device, its performance set is Calculate volatility: ;in, This indicates the total number of data points from the verification devices. This represents the performance metric of the data from the m-th verification device. This metric reflects the performance differences between data from different devices. A larger difference indicates uneven model generalization. A volatility threshold is set for the server. ; The independent verification device dataset is built by the server and needs to specifically cover typical challenging device data in smart home scenarios, such as cross-device intent recognition, i.e., the response of the same command on different types of devices, state prediction in low-resource situations, inferring the overall environment based on only a few sensors, and robustness testing against noise and missing data. By monitoring the performance fluctuations of the model on such heterogeneous validation sets, its generalization ability and stability in complex smart home environments can be more accurately reflected. The volatility is calculated by the dispersion of the evaluation scores of each device data in the validation device dataset. The evaluation score can be one of the following indicators: accuracy, loss value, or ranking. One indicator is always used in the same implementation process. The server sets a volatility threshold. When the volatility is higher than the volatility threshold, it is determined that the performance difference of the model on different device data has increased. It is necessary to strengthen conflict update suppression or adjust the boundary of the collaborative group. The volatility threshold is calculated by the server from the historical volatility statistics and is fixed once in each training phase. After each training round, the server checks all collaborative groups. With the whole The independent verification device dataset is maintained by the server. Its device data design reflects the typical intent recognition, state prediction or anomaly detection needs of smart home scenarios, and covers different device types and data quality conditions. Consistency threshold With volatility threshold It is not a fixed value; the server adjusts it based on recent data after each round of aggregation. and Historical data, such as the last five rounds, is adaptively adjusted using moving averages or quantile statistics to match the stability requirements of different training stages. There are three scenarios: Situation A: and This indicates high collaboration within the group and low performance differences of the model on data from different devices, indicating stable convergence. The server will maintain the current sampling and aggregation strategy in the next round. Situation B: and This indicates that the conflicts within the collaborative group have not been completely eliminated, but the overall performance is acceptable. The server can appropriately adjust the sampling constraints, such as increasing the difficulty normalization strength, and repeat steps S2 and S3 to continue the iteration. Case C: This indicates that the performance fluctuations between different devices are large, which may be due to updates in certain collaboration groups or certain terminals causing a decline in the performance of the global model. In this case, we should return to step S2 to re-evaluate the performance of each terminal. and Adjust the scheme for dividing the collaborative groups, and strengthen the suppression of conflict updates in step S3, or even adopt cluster training; Executed after each iteration of federated learning, if consecutive If condition A is met on average in each round, the federated model is considered to have converged stably, and iteration can be stopped or the aggregation frequency reduced. If condition C occurs consecutively before the preset maximum number of iteration rounds, it indicates that the current strategy is difficult to converge, and a more complex model or a more robust personalized solution can be considered. The server can also dynamically adjust the threshold during the iteration process. and To adapt to the convergence speed at different stages; The number of consecutive rounds is preset by the server as the stopping criterion window length and written into the training configuration at the start of training. When the situation remains stable within the stopping criterion window length, the server determines that iteration can be stopped or the aggregation frequency can be reduced. Once the model converges, the server outputs the final global model parameters or multiple cluster model parameters, and records the parameters of each terminal throughout the training process. , , , These results serve as the basis for auditing and optimization. Through these records, the contribution of different terminals to the final model can be evaluated, and potential abnormal terminals can also be identified. It enables dynamic monitoring of the cross-terminal small sample collaborative training process, ensuring training stability and generalization ability, and timely adjustment of strategies or termination of iteration based on monitoring indicators. The dual-criteria mechanism can both suppress model fluctuations and avoid inefficiency caused by over-adjustment.

[0018] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0019] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, in the form of a computer program product.

[0020] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and inventive constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0021] In addition, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.

[0022] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0023] In conclusion, the above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for cross-terminal collaborative training of few-shot models based on federated learning, characterized in that, Includes steps; Step S1: Collect interactive data fragments and context information of smart home devices, perform alignment, normalization and desensitization processing on the data of different devices, encode the samples into vector representation through the edge embedding model, calculate the class centroid, intra-class dispersion and inter-class separability of the data of each device, and statistically analyze the signal quality mean vector, feedback confidence mean, gradient mean vector and gradient norm mean and variance, generate device data summary and upload it to the federated server; Step S2: The federated server normalizes the device data summary, calculates the device data difficulty index and conflict prior index. The device data difficulty index is calculated by intra-class dispersion and inter-class separability. The conflict prior index is calculated by the similarity and deviation angle of the terminal update direction relative to the reference direction. The server sets the boundary of the collaborative group, conflict threshold, interference threshold, consistency threshold and volatility threshold, compares the index with the threshold to determine the heterogeneous collaborative group identifier, and sends the device data construction configuration to constrain terminal sampling. Step S3: Calculate the cluster optimization baseline and conflict score within the group. If the conflict score exceeds the conflict threshold, perform a delay or shadow verification on the update. Calculate the cross-cooperative group intersection interference matrix and compare it with the interference threshold to determine isolation aggregation or cluster aggregation. Generate the device contribution weight and update the model based on the conflict score, device data difficulty index, signal quality mean vector and feedback confidence mean. Step S4: At the end of each round, calculate the intra-group synergy score and verify the volatility. If the consistency score is not less than the consistency threshold and the volatility is not greater than the volatility threshold, proceed to the next round; otherwise, return to step S2 to adjust the configuration or end the iteration, and output the model parameters and aggregated explanation log.

2. The method for cross-terminal collaborative training of few-shot models based on federated learning according to claim 1, characterized in that, The terminal constructs small sample learning device data, which includes: preprocessing, labeling and augmenting local data, and dividing it into support set and query set; The generated device data summary includes: calculating and recording the category coverage of the device data, the intra-class dispersion and inter-class separability of each category, the signal quality vector, the feedback confidence, and the gradient statistics during the local few-sample training process.

3. The method for cross-terminal collaborative training of few-shot models based on federated learning according to claim 2, characterized in that, Intra-class dispersion is obtained by calculating the average distance from the vector representation of all supporting samples of the same class to the centroid of that class; Inter-class separability is obtained by calculating the distance between the centroids of different classes; The device data summary should include at least the class coverage vector, intra-class dispersion vector, inter-class separability matrix, signal quality mean vector, feedback confidence mean, gradient norm mean and variance, and should be reported through an encrypted channel.

4. The method for cross-terminal collaborative training of few-shot models based on federated learning according to claim 2, characterized in that, Before generating a device data summary, the terminal will perform multiple rounds of small-sample training locally to statistically analyze the gradient norm and its fluctuations during the training process. The signal quality vector includes one or more of the following: speech signal-to-noise ratio, data missing rate, sensor drift intensity, and network latency fluctuation. Feedback confidence is determined by explicit user feedback, implicit feedback, and cross-session consistency.

5. The method for cross-terminal collaborative training of few-shot models based on federated learning according to claim 1, characterized in that, When calculating the difficulty index of device data, the federal server comprehensively considers the overall level of intra-class dispersion, the overall level of inter-class separability, and the gradient variance reported by each terminal. When calculating the conflict prior index, the deviation between the terminal update gradient direction and the global average direction, the mean gradient norm, and the feedback confidence level are taken into account.

6. The method for cross-terminal collaborative training of few-shot models based on federated learning according to claim 5, characterized in that, Based on the calculated device data difficulty index and conflict prior index, the federated server divides the terminals into different cooperative groups and generates targeted device data sampling constraints for each cooperative group. The sampling constraints include one or more of the following: support set size, sampling weights for each class of samples, and low-quality sample suppression coefficient, which are used to guide the terminal to adjust the local device data construction in the next round of training.

7. The method for cross-terminal collaborative training of few-shot models based on federated learning according to claim 1, characterized in that, Gradient direction conflict analysis includes: calculating a cluster optimization baseline for each heterogeneous collaborative group; For each terminal update within the collaboration group, calculate the cosine of the direction angle between the update and the cluster optimization baseline, and calculate the conflict score based on the cosine value and the relative magnitude of the update within the collaboration group.

8. The method for cross-terminal collaborative training of few-shot models based on federated learning according to claim 7, characterized in that, The adaptive weighted aggregation strategy is as follows: based on the conflict score of the terminal, the device data difficulty index, and the conflict prior index, a device contribution weight is assigned to each terminal update, and the weight is negatively correlated with the conflict score. The clustering and aggregation strategy is as follows: when a serious update direction conflict is identified between different heterogeneous collaborative groups, the terminals in the conflicting collaborative groups are re-clustered, and independent models are generated for each cluster.

9. The method for cross-terminal collaborative training of few-shot models based on federated learning according to claim 1, characterized in that, The intra-group collaboration score is calculated based on the average similarity between the update directions of all terminals within the collaboration group; The calculation of global model performance volatility is based on the degree of dispersion of the performance metrics exhibited by the model on a set of independent validation device datasets; The dual criteria are a preset consistency threshold and a performance volatility threshold.

10. The method for cross-terminal collaborative training of few-shot models based on federated learning according to claim 9, characterized in that, The rules for dynamic decision-making based on dual criteria include: if the consistency score is higher than its threshold and the performance volatility is lower than its threshold, then the current strategy is maintained and training continues. If the consistency score is lower than its threshold, a return to step S2 is triggered to adjust the device data sampling constraints or re-divide the collaboration groups; If the performance volatility exceeds its threshold, return steps S2 and S3 are triggered to strengthen conflict suppression or enable cluster aggregation.

Images