A data fusion method under resource-limited conditions based on federated computing

Abstract

This application discloses a data fusion method based on federated computing under resource-constrained conditions, belonging to the field of network inference. The method includes: generating inference upload data at the user terminal and sending it to proxy training nodes; the proxy training nodes training their local models based on the inference upload data and uploading the training data to a training aggregation node; the training aggregation node aggregating multiple training data sets to generate fused training data, which is then distributed to each proxy training node. Finally, the proxy training nodes forward the fused training data to the user terminal to update the local model. This addresses the problem of rapid model optimization under resource-constrained conditions on the terminal.

Description

Technical Field

[0001] This application belongs to the field of network reasoning, and specifically relates to a data fusion method, system, deployment method, apparatus, storage medium, device and computer program product based on federated computing under resource-constrained conditions. Background Technology

[0002] In resource-constrained scenarios such as the Internet of Things (IoT), in-vehicle systems, and smart security, terminal devices are typically deployed at the network edge to handle data collection and inference tasks. These devices possess certain sensing and computing capabilities, enabling them to perform lightweight model inference operations locally, thereby achieving low-latency response and privacy protection.

[0003] Currently, the iteration mechanism of terminal models typically adopts a centralized training architecture, where the raw data collected by the terminals is uploaded to a central server, which then uniformly completes model training and updates before distributing the updated models to each terminal. Under conditions of sufficient central computing resources and stable network connectivity, this approach can achieve high model accuracy and is suitable for some centralized deployment scenarios.

[0004] However, although inference requests on the terminal side often reflect the user's latest needs and environmental conditions, the limited resources of terminal devices make it difficult to undertake local model training tasks. This prevents the aforementioned centralized solutions from achieving rapid model updates based on this inference data. This not only wastes the timeliness of inference data but also limits the model's adaptability to dynamic environments, making it difficult to meet the actual needs for rapid model iteration and continuous optimization. Summary of the Invention

[0005] This application aims to provide a data fusion method, system, deployment method, apparatus, storage medium, device, and computer program product based on federated computing under resource-constrained conditions, which at least solves the problem of difficulty in updating models using real-time information when terminal computing resources are limited.

[0006] In a first aspect, embodiments of this application disclose a data fusion method based on federated computing under resource-constrained conditions, applied to a local user terminal, including: Inference upload data is generated based on the inference input data input by the user, and the inference upload data is sent to the target agent training node; Receive fused training data forwarded from the target agent training node, and update the local terminal model based on the fused training data.

[0007] Secondly, embodiments of this application also disclose a data fusion method based on federated computing under resource-constrained conditions, applied to a local agent training node, including: In response to inference upload data sent from the target user terminal, the local agent training model is trained based on the inference upload data to obtain local training data; The local training data is uploaded to the training aggregation node; after receiving the training data sent from multiple proxy training nodes, the training aggregation node generates fused training data based on the multiple training data and distributes the fused training data to each proxy training node. Upon receiving the fused training data, the fused training data is forwarded to the target user terminal.

[0008] Thirdly, embodiments of this application disclose a data fusion method based on federated computing under resource-constrained conditions, applied to training aggregation nodes, including: Receive training data sent from multiple proxy training nodes, and generate fused training data based on the multiple training data; The fused training data is distributed to each proxy training node; the proxy training node receiving the fused training data is used to forward the fused training data to the target user terminal; the target user terminal is the user terminal that sends target inference upload data to the proxy training node; the target inference upload data is used to generate training data sent from the proxy training node.

[0009] Fourthly, embodiments of this application also disclose a data fusion system based on federated computing under resource-constrained conditions, comprising: Multiple user terminals, multiple agent training nodes, and a training aggregation node; each user terminal is communicatively connected to at least one of the agent training nodes, and the agent training nodes are communicatively connected to the training aggregation node; The user terminal is used to generate inference upload data based on inference input data input by the user, send the inference upload data to the target agent training node, receive fused training data forwarded from the target agent training node, and update the local terminal model according to the fused training data. The proxy training node is used to respond to inference upload data sent from the target user terminal, train the local proxy training model based on the inference upload data to obtain local training data, upload the local training data to the training aggregation node, and forward the fused training data to the target user terminal when the fused training data is received. The training aggregation node is used to receive training data sent from multiple proxy training nodes, generate fused training data based on the multiple training data, and distribute the fused training data to each proxy training node.

[0010] Fifthly, embodiments of this application also disclose a deployment method for a data fusion system under resource-constrained conditions based on federated computing, including: The user terminal-agent training node relationship requirement of the data fusion system based on federated computing under resource-constrained conditions, as described in the fourth aspect, is taken as the first constraint, the number of agent training nodes is taken as the second constraint, and the agent training node-training aggregation node relationship requirement is taken as the third constraint. The optimization objective is to minimize the resource consumption of the data fusion system based on federated computing under resource-constrained conditions. An optimization function for the data fusion system based on federated computing under resource-constrained conditions is established. Solve the optimization function to obtain the deployment parameters for the resource-constrained data fusion system based on federated computing, and deploy the resource-constrained data fusion system based on federated computing according to the deployment parameters.

[0011] Sixthly, embodiments of this application also disclose a data fusion device based on federated computing under resource-constrained conditions, applied to a local user terminal, including: The input upload module is used to generate inference upload data based on inference input data input by the user, and send the inference upload data to the target agent training node; The model download module is used to receive fused training data forwarded from the target agent training node and update the local terminal model according to the fused training data.

[0012] Seventhly, embodiments of this application also disclose a data fusion apparatus under resource-constrained conditions based on federated computing, applied to a local agent training node, comprising: The local training module is used to train the local agent training model based on the inference upload data sent from the target user terminal in response to the inference upload data, so as to obtain local training data. The training upload module is used to upload the local training data to the training aggregation node; after receiving the training data sent from multiple proxy training nodes, the training aggregation node generates fused training data based on the multiple training data and distributes the fused training data to each proxy training node. The model forwarding module is used to forward the fused training data to the target user terminal upon receiving the fused training data.

[0013] Eighthly, embodiments of this application also disclose a data fusion apparatus under resource-constrained conditions based on federated computing, applied to training aggregation nodes, including: The training fusion module is used to receive training data sent from multiple proxy training nodes and generate fused training data based on the multiple training data. The model distribution module is used to distribute the fused training data to each proxy training node; the proxy training node that receives the fused training data is used to forward the fused training data to the target user terminal; the target user terminal is the user terminal that sends target inference upload data to the proxy training node; the target inference upload data is used to generate training data sent from the proxy training node.

[0014] Ninthly, embodiments of this application also disclose a deployment apparatus for a data fusion system under resource-constrained conditions based on federated computing, comprising: The modeling module is used to establish an optimization function for the data fusion system under resource-constrained conditions based on federated computing, with the user terminal-agent training node relationship requirement as the first constraint, the number of agent training nodes as the second constraint, and the agent training node-training aggregation node relationship requirement as the third constraint, and with the optimization objective of minimizing the resource consumption of the data fusion system under resource-constrained conditions based on federated computing. The configuration solution module is used to solve the optimization function to obtain the deployment parameters for the resource-constrained data fusion system based on federated computing, and to deploy the resource-constrained data fusion system based on federated computing according to the deployment parameters.

[0015] In a tenth aspect, embodiments of this application also disclose a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps described in any one of the first, second, third, and fifth aspects.

[0016] Eleventhly, embodiments of this application also disclose an electronic device, including a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, performs the steps as described in any one of the first, second, third, and fifth aspects.

[0017] In a twelfth aspect, embodiments of this application also disclose a computer program product on which a computer program is stored, and which, when executed by a processor, implements the steps described in any one of the first, second, third, and fifth aspects.

[0018] In summary, in this embodiment, the user terminal first sends inference upload data reflecting the user's latest needs and environmental status to the target proxy training node. This allows the proxy training node to process the data as training material, ensuring the timeliness of data utilization and avoiding training lag caused by delayed uploads or post-processing, effectively enhancing the model's adaptability to dynamic environments. Furthermore, after receiving the inference upload data, the proxy training node performs local training based on the data and generates local training data, which is then uploaded to the training aggregation node. This not only alleviates the technical bottleneck of terminal devices being unable to perform local training due to limited computing power but also achieves effective communication between data and the model through the relay training mechanism of the proxy node. The coupling mechanism reduces the resource burden on terminals during training while improving the system's scalability and deployment flexibility. Then, after the training aggregation node receives training data from multiple proxy training nodes, it generates fused training data with global data attributes through data fusion. This fully integrates data features from different terminal environments, improving the model's generalization ability and overall accuracy, and reducing training bias caused by data silos or environmental differences. Finally, the proxy training nodes synchronize the model results from the fused training data to the corresponding user terminals, achieving closed-loop model iteration driven by inference data. This allows the terminal model to quickly respond to changes in user behavior, further improving inference accuracy and system response efficiency while ensuring inference performance. Therefore, based on the method of this application embodiment, by constructing a hierarchical structure of "user terminal—proxy training node—training aggregation node," it achieves efficient utilization of inference data and continuous model optimization under resource-constrained conditions. This solves the problem of model update lag caused by the inability of terminals to train locally and the waste of inference data timeliness in existing centralized training architectures, improving the adaptability, training efficiency, and deployment flexibility of the "Magic Inference" system. Attached Figure Description

[0019] In the attached diagram: Figure 1 This is a flowchart illustrating the steps of a data fusion method based on federated computing under resource-constrained conditions, as provided in an embodiment of this application. Figure 2 This is a flowchart of another data fusion method based on federated computing under resource-constrained conditions provided in the embodiments of this application; Figure 3 This is a topology diagram of a data fusion system based on federated computing under resource-constrained conditions, provided in an embodiment of this application. Figure 4 This is a flowchart illustrating the steps of a deployment method for a data fusion system based on federated computing under resource-constrained conditions, as provided in an embodiment of this application. Figure 5This is a block diagram of a data fusion device based on federated computing under resource-constrained conditions, provided in an embodiment of this application. Figure 6 This is a block diagram of another data fusion device based on federated computing under resource-constrained conditions provided in the embodiments of this application; Figure 7 This is a block diagram of another data fusion device based on federated computing under resource-constrained conditions, provided in the embodiments of this application; Figure 8 This is a block diagram of a deployment device for a data fusion system based on federated computing under resource-constrained conditions, provided in an embodiment of this application. Figure 9 This is a block diagram of an electronic device provided in one embodiment of this application. Detailed Implementation

[0020] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0021] The terms "first," "second," etc., used in this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class, without limiting the number of objects; for example, a first object can be one or more. Furthermore, "and / or" in this application indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects have an "or" relationship.

[0022] like Figure 1 The image shows a data fusion method based on federated computing under resource-constrained conditions, provided in an embodiment of this application.

[0023] Steps 101 and 107 are applied to local user terminals; steps 102, 103, and 106 are applied to local agent training nodes; and steps 104 and 105 are applied to training aggregation nodes.

[0024] The method may include the following steps: Step 101: Generate inference upload data based on the inference input data input by the user, and send the inference upload data to the target agent training node.

[0025] In some embodiments of this application, to enable the terminal device to promptly incorporate its latest collected environmental data into the federated training process, inference upload data is generated from user-inputted inference input data and sent to the target agent training node. Specifically, after receiving user-inputted inference input data, such as image, audio, or sensor data, the terminal device converts it into a structured inference upload data format and then sends it to the agent training node bound to the terminal via a network interface. Inference upload data refers to representative data samples generated by the terminal during the execution of inference tasks, reflecting the current environmental state or user behavior characteristics. This allows inference data from the terminal to participate in the model optimization process in real time, thereby improving the timeliness of data utilization and enhancing the model's adaptability to dynamic environments.

[0026] In a specific example, a user types the question into the smart security terminal's interface: "Can the abnormal behavior detected by the camera be used to train the model?" Upon receiving this question, the system correlates the semantic information entered by the user with the camera's current inference results, extracts video frame data containing abnormal behavior tags, and encapsulates it as inference upload data. The terminal device then sends this inference upload data to the proxy training node in the home gateway. Upon receiving this data, the proxy training node uses it in subsequent local model training processes. In this way, the system can respond promptly to the user's inference needs and introduce key data into the federated training process, providing fundamental data support for continuous model optimization.

[0027] Step 102: In response to the inference upload data sent from the target user terminal, train the local agent training model based on the inference upload data to obtain local training data.

[0028] In some embodiments of this application, to enable the proxy training node to fully utilize the inference data uploaded by the terminal for model optimization, after receiving the inference data uploaded by the target user terminal, the local proxy training model is trained based on this data to obtain local training data. Specifically, after receiving the inference data uploaded, the proxy training node uses this data as training samples to perform model training tasks locally, update model parameters, and generate local training data. In this way, the proxy training node can complete the initial training of the model without relying on the terminal's computing power, alleviating the training bottleneck caused by limited terminal resources, and providing high-quality local training data for subsequent global model fusion.

[0029] In a specific example, a user inputs the question into a smart home terminal: "Why has my voice assistant been making more recognition errors lately?" The system processes this voice input as inference input data and converts it into inference upload data, which is then sent to the agent training node in the home gateway. Upon receiving this data, the agent training node, combined with historical speech recognition logs, fine-tunes the speech recognition model locally, generating new local training data. In this way, the system can quickly adjust model parameters based on user-feedback inference data, providing more representative training samples for subsequent global model fusion, thereby improving the voice assistant's recognition accuracy in this home environment.

[0030] Step 103: Upload the local training data to the training aggregation node.

[0031] In some embodiments of this application, to enable the training aggregation node to collect model update information from different proxy training nodes, local training data is uploaded to the training aggregation node. Specifically, after completing local model training, the proxy training node sends the generated local training data to a preset training aggregation node via a network interface. Local training data refers to the parameter update results or gradient information generated by the proxy training node after completing model training based on the inference-uploaded data, which is used to participate in subsequent global model fusion. In this way, the training aggregation node can gather training results from multiple proxy training nodes, providing the necessary data foundation for subsequent data fusion and model aggregation, thereby improving the model's generalization ability and overall accuracy.

[0032] In a specific example, a user inputs the question into a smart security terminal: "Can the camera recognition model continuously optimize based on my home environment?" The system uploads the inference data associated with this question to a proxy training node in the home gateway. After completing local model training, the proxy node generates local training data containing environmental features. Subsequently, the proxy training node uploads this training data to a training aggregation node on the community server. In this way, the system can incorporate the model training results from the user's home environment into the community-level model optimization process, providing diverse data sources for subsequent fusion training, thereby improving the model's recognition accuracy in different home scenarios.

[0033] Step 104: Receive training data sent from multiple proxy training nodes, and generate fused training data based on the multiple training data.

[0034] In some embodiments of this application, to achieve data fusion and model optimization across terminal environments, a training aggregation node receives training data sent from multiple proxy training nodes and generates fused training data based on the multiple training data. Specifically, the training aggregation node receives local training data uploaded from different proxy training nodes through a preset communication interface, aggregates this data, and generates fused training data with global data attributes. Fusion training data refers to a unified training result formed by aggregating the local training results of multiple proxy training nodes, used to improve the model's generalization ability and overall accuracy. In this way, the system can integrate data features from multiple terminal environments, alleviate the data silo problem, reduce training bias caused by environmental differences, and provide a unified data foundation for subsequent model updates.

[0035] In a specific example, a user enters the question into a smart medical terminal: "Can diagnostic models from different hospitals share training results?" The system associates this question with proxy training nodes in the hospital gateway. Multiple hospital proxy nodes upload model parameters trained on local case data to the training aggregation node of the regional medical center. Upon receiving this training data, the training aggregation node performs a weighted average operation to generate fused training data, which is then used to update the regional diagnostic model. In this way, the system can integrate the training results from multiple medical institutions, improving the model's diagnostic accuracy and adaptability in diverse patient environments.

[0036] Step 105: Distribute the fused training data to each agent training node.

[0037] In some embodiments of this application, to ensure that each proxy training node can promptly obtain the fused model update results, fused training data is distributed to each proxy training node. Specifically, after generating the fused training data, the training aggregation node sends the fused training data to all participating proxy training nodes through a preset distribution mechanism. Fusion training data refers to unified model parameters or training results formed by aggregating local training data uploaded by multiple proxy training nodes, used to synchronously update the local model state of each proxy node. This system ensures that each proxy training node obtains consistent model optimization results within the same training cycle, thereby improving the model's collaborative consistency and overall performance.

[0038] In a specific example, a user enters the question "Can my speech recognition model stay synchronized with other devices?" into a smart office terminal. The system associates this question with a proxy training node in the enterprise gateway. After completing local training, this node uploads the data to a training aggregation node on the enterprise server. The training aggregation node receives training data from multiple office terminals, generates fused training data, and distributes this data to all proxy training nodes. Ultimately, the system enables the speech recognition models on all office terminals to maintain a consistent update status, thereby improving the accuracy and collaborative response capabilities of cross-device speech recognition.

[0039] Step 106: Upon receiving the fused training data, forward the fused training data to the target user terminal.

[0040] In some embodiments of this application, to enable the target user terminal to promptly obtain the fused training results and complete local model updates, the fused training data will be forwarded to the target user terminal upon receipt by the proxy training node. Specifically, after receiving the fused training data from the training aggregation node via the communication interface, the proxy training node forwards the data to the corresponding user terminal based on the terminal identification information. The fused training data refers to the unified model parameters or training results generated by the training aggregation node based on the local training data uploaded by multiple proxy training nodes, used to improve the adaptability and inference accuracy of the terminal model. This system enables rapid synchronization of model update results on the terminal side, constructing a closed-loop iterative mechanism driven by inference data, thereby improving the terminal model's responsiveness to changes in user behavior.

[0041] In a specific example, a user enters the question into their smart wearable device: "Can my motion recognition model be adjusted based on recent training data?" The system associates this question with a proxy training node in the home gateway. Upon receiving the fused training data from the training aggregation node, this node identifies the corresponding terminal device and forwards it to the user's smart bracelet. The smart bracelet then automatically updates its model parameters. In this way, the system can quickly synchronize the fused training results to the user's terminal, enabling the terminal model to better adapt to the user's recent motion behavior characteristics, thereby improving recognition accuracy and personalized response capabilities.

[0042] Step 107: Receive the fused training data forwarded from the target agent training node, and update the local terminal model based on the fused training data.

[0043] In some embodiments of this application, to enable the user terminal to continuously optimize its local model to adapt to the latest environmental changes, the local terminal model is updated based on the fused training data forwarded by the target proxy training node. Specifically, after receiving the fused training data forwarded by the proxy training node through the communication module, the user terminal inputs this data as a parameter into the local model update module to adjust the model weights or structure. Fusion training data refers to unified model parameters or training results generated by the training aggregation node based on the local training data uploaded by multiple proxy training nodes, used to improve the adaptability and inference accuracy of the terminal model. In this way, the terminal device can complete model updates without participating in model training, thereby achieving continuous optimization of inference performance and enhanced environmental responsiveness.

[0044] In a specific example, a user inputs the question "Can my speech recognition model automatically adapt to recent pronunciation changes?" into the smart voice assistant terminal. The system associates this question with the agent training node in the home gateway. After receiving the fused training data, this node forwards it to the user's voice assistant terminal. Upon receiving this data, the voice assistant terminal automatically executes the model update process, adjusting the parameter configuration of the recognition model. Ultimately, the system enables the voice assistant to optimize the model based on recent user pronunciation characteristics, thereby improving the accuracy and response speed of speech recognition.

[0045] In summary, in this embodiment, the user terminal first sends inference upload data reflecting the user's latest needs and environmental status to the target proxy training node. This allows the proxy training node to process the data as training material, ensuring the timeliness of data utilization and avoiding training lag caused by delayed uploads or post-processing, effectively enhancing the model's adaptability to dynamic environments. Furthermore, after receiving the inference upload data, the proxy training node performs local training based on the data and generates local training data, which is then uploaded to the training aggregation node. This not only alleviates the technical bottleneck of terminal devices being unable to perform local training due to limited computing power but also achieves effective communication between data and the model through the relay training mechanism of the proxy node. The coupling mechanism reduces the resource burden on terminals during training while improving the system's scalability and deployment flexibility. Then, after the training aggregation node receives training data from multiple proxy training nodes, it generates fused training data with global data attributes through data fusion. This fully integrates data features from different terminal environments, improving the model's generalization ability and overall accuracy, and reducing training bias caused by data silos or environmental differences. Finally, the proxy training nodes synchronize the model results from the fused training data to the corresponding user terminals, achieving closed-loop model iteration driven by inference data. This allows the terminal model to quickly respond to changes in user behavior, further improving inference accuracy and system response efficiency while ensuring inference performance. Therefore, based on the method of this application embodiment, by constructing a hierarchical structure of "user terminal—proxy training node—training aggregation node," it achieves efficient utilization of inference data and continuous model optimization under resource-constrained conditions. This solves the problem of model update lag caused by the inability of terminals to train locally and the waste of inference data timeliness in existing centralized training architectures, improving the adaptability, training efficiency, and deployment flexibility of the "Magic Inference" system.

[0046] Figure 2 This application provides another data fusion method based on federated computing under resource-constrained conditions. Steps 201, 207, and 208 are applied to local user terminals; steps 202, 203, and 206 are applied to local agent training nodes; and steps 204 and 205 are applied to training aggregation nodes.

[0047] The method may include the following steps: Step 201: Generate inference upload data based on the inference input data input by the user, and send the inference upload data to the target agent training node.

[0048] The method shown in this step has been explained in step 101 and will not be repeated here.

[0049] Optionally, step 201 involves determining the user-inputted inference input data and / or the summary data of the inference input data as inference upload data, thereby generating inference upload data based on the user-inputted inference input data.

[0050] In some embodiments of this application, to enable the terminal device to effectively convert user-inputted inference data into a data format usable for training, the user-inputted inference input data and / or its summary data are identified as inference upload data, thus realizing the process of generating inference upload data based on user-inputted inference input data. Specifically, after receiving the user-inputted raw inference data, the terminal device selects to directly use the raw data or extracts a summary (such as feature vector compression, label annotation, time segment truncation, etc.) according to a preset strategy to generate structured inference upload data. The summary data refers to representative information fragments extracted from the raw inference input data, used for model training without exposing the complete data content. By performing this step, the system can flexibly generate data formats suitable for proxy training nodes to process, while balancing data privacy and communication efficiency, thereby improving the controllability of data upload and the quality of training data.

[0051] In a specific example, a user inputs the question into a smart voice assistant terminal: "Can what I just said be used to improve the recognition model?" Upon receiving this voice input, the system extracts the audio segment as the raw inference input data and generates two candidate data sets based on a configured strategy: a complete audio file and a speech feature summary of the audio (such as a Mel spectrogram or acoustic embedding vector). Depending on network bandwidth and privacy settings, the system chooses to send the speech summary as the inference upload data to the proxy training node. In this way, the system protects the user's voice privacy while ensuring the representativeness and transmission efficiency of the training data, providing fundamental data support for subsequent model optimization.

[0052] Step 202: In response to the inference upload data sent from the target user terminal, train the local agent training model based on the inference upload data to obtain local training data.

[0053] The method shown in this step has been explained in step 102 and will not be repeated here.

[0054] Optionally, in order to train the local agent training model based on the inference uploaded data to obtain local training data, step 202 includes the following sub-steps: Sub-step 2021 involves performing subgradient iteration on the loss function of the local agent training model based on the inference uploaded data to obtain the trained local agent training model.

[0055] In some embodiments of this application, to enable the proxy training node to optimize local model parameters based on inference upload data, subgradient iteration is performed on the loss function of the local proxy training model to obtain the trained local proxy training model. Specifically, after receiving inference upload data, the proxy training node uses it as input samples, calculates the loss function of the current model, and updates the model parameters using a subgradient iteration method. Subgradient iteration is a strategy for optimization when the objective function is non-differentiable or non-convex, suitable for scenarios where complex gradient calculations cannot be performed on resource-constrained devices. In this way, the proxy training node can complete the initial optimization of model parameters locally, providing a stable model foundation for subsequent extraction and global fusion of training data.

[0056] In a specific example, a user inputs the question into a smart security terminal: "Can the camera recognition model continuously optimize based on my home environment?" The system uploads the inference data associated with this question to the agent training node in the home gateway. Upon receiving this data, the agent node constructs a local loss function and updates the model parameters using a subgradient iterative method. This iterative process is completed on the gateway device, which has fewer resource constraints, ensuring sufficient computing resources. Ultimately, the agent training node obtains a locally trained agent model optimized for the user's home environment, laying the foundation for subsequent training data extraction and model fusion.

[0057] Sub-step 2022: Extract local training data from the trained local agent training model.

[0058] In some embodiments of this application, in order to transform the trained model results into standardized data that can be used for subsequent aggregation, local training data is extracted from the trained local proxy training model. Specifically, after the proxy training node completes model training, it extracts training products such as parameter update results, gradient information, or feature embeddings from the model and organizes them into a structured local training data format. Local training data refers to the data content exported from the model after the proxy node completes model training and that can be used to participate in global fusion, typically including model weights, bias terms, loss values, feature representations, etc. In this way, the system can transform the model training results into a standardized data structure, which is convenient for subsequent uploading to the training aggregation node for fusion processing, thereby achieving continuity in the model optimization process and consistency in the data interface.

[0059] In a specific example, a user inputs the question into the smart vehicle terminal: "Can my driving behavior be learned and fed back by the model?" The system associates this question with the agent training node in the vehicle gateway. After receiving the inference upload data, this node completes model training and extracts the parameter update results and embedding vectors containing driving behavior features from the trained model, organizing them into local training data. In this way, the system transforms the user's driving behavior features into a data format that can participate in federated training, providing structured data support for subsequent model fusion and feedback mechanisms.

[0060] Step 203: Upload the local training data to the training aggregation node.

[0061] The method shown in this step has been explained in step 103 and will not be repeated here.

[0062] Step 204: Receive training data sent from multiple proxy training nodes, and generate fused training data based on the multiple training data.

[0063] The method shown in this step has been explained in step 104 and will not be repeated here.

[0064] Optionally, step 204 involves aggregating multiple training data to generate fused training data.

[0065] In some embodiments of this application, to achieve data fusion and model optimization across terminal environments, a process of generating fused training data is implemented by aggregating multiple training data sets. Specifically, after receiving local training data uploaded from multiple proxy training nodes via a communication interface, the training aggregation node uses an aggregation strategy (such as weighted averaging, gradient merging, or model parameter fusion) to process the data uniformly, generating fused training data with global data attributes. Fusion training data refers to the unified training result formed by aggregating the local training results of multiple proxy training nodes, used to improve the model's generalization ability and overall accuracy. In this way, the system can integrate data features from multiple terminal environments, alleviate the data silo problem, reduce training bias caused by environmental differences, and provide a unified data foundation for subsequent model updates.

[0066] In a specific example, a user enters the question into a smart medical terminal: "Can diagnostic models from different hospitals share training results?" The system associates this question with proxy training nodes in the hospital gateway. Multiple hospital proxy nodes upload model parameters trained on local case data to the training aggregation node of the regional medical center. Upon receiving this training data, the training aggregation node uses a weighted averaging strategy to aggregate the model parameters uploaded by each hospital, generating fused training data, which is then used to update the regional diagnostic model. Ultimately, the system can integrate the training results from multiple medical institutions, improving the model's diagnostic accuracy and adaptability in diverse patient environments.

[0067] Step 205: Distribute the fused training data to each agent training node.

[0068] The method shown in this step has been explained in step 105 and will not be repeated here.

[0069] Step 206: Upon receiving the fused training data, forward the fused training data to the target user terminal.

[0070] The method shown in this step has been explained in step 106 and will not be repeated here.

[0071] Step 207: Receive the fused training data forwarded from the target agent training node, and update the local terminal model based on the fused training data.

[0072] The method shown in this step has been explained in step 107 and will not be repeated here.

[0073] Step 208: Input the inference input data into the updated local terminal model to perform the inference task.

[0074] In some embodiments of this application, to enable the user terminal to complete inference tasks based on the latest model state, inference input data is input into the updated local terminal model to execute the inference task. Specifically, after completing the model update, the terminal device transmits the user-inputted inference data as input to the local model inference module, performs inference calculations, and outputs the results. Inference input data refers to the original data samples actively provided by the user on the terminal side to trigger model inference, such as images, voice, text, or sensor data. In this way, the terminal device can use the latest model parameters to complete the inference task, thereby improving the accuracy and response efficiency of the inference results and realizing a closed loop for practical applications after model iteration.

[0075] In a specific example, a user inputs the question "Can you recognize my 'turn on the lights' now?" into the smart voice assistant terminal. The system uses this voice command as input data for inference, feeding it into the newly updated speech recognition model for processing. The terminal model recognizes the control intent corresponding to the voice command and triggers the home lighting system to turn on the lights. Through this execution process, the system verifies the model's updated recognition capabilities, achieving a closed-loop linkage between user behavior and model inference, thereby improving the voice assistant's responsiveness and interactive experience in real-world scenarios.

[0076] In summary, in this embodiment, the user terminal first sends inference upload data reflecting the user's latest needs and environmental status to the target proxy training node. This allows the proxy training node to process the data as training material, ensuring the timeliness of data utilization and avoiding training lag caused by delayed uploads or post-processing, effectively enhancing the model's adaptability to dynamic environments. Furthermore, after receiving the inference upload data, the proxy training node performs local training based on the data and generates local training data, which is then uploaded to the training aggregation node. This not only alleviates the technical bottleneck of terminal devices being unable to perform local training due to limited computing power but also achieves effective communication between data and the model through the relay training mechanism of the proxy node. The coupling mechanism reduces the resource burden on terminals during training while improving the system's scalability and deployment flexibility. Then, after the training aggregation node receives training data from multiple proxy training nodes, it generates fused training data with global data attributes through data fusion. This fully integrates data features from different terminal environments, improving the model's generalization ability and overall accuracy, and reducing training bias caused by data silos or environmental differences. Finally, the proxy training nodes synchronize the model results from the fused training data to the corresponding user terminals, achieving closed-loop model iteration driven by inference data. This allows the terminal model to quickly respond to changes in user behavior, further improving inference accuracy and system response efficiency while ensuring inference performance. Therefore, based on the method of this application embodiment, by constructing a hierarchical structure of "user terminal—proxy training node—training aggregation node," it achieves efficient utilization of inference data and continuous model optimization under resource-constrained conditions. This solves the problem of model update lag caused by the inability of terminals to train locally and the waste of inference data timeliness in existing centralized training architectures, improving the adaptability, training efficiency, and deployment flexibility of the "Magic Inference" system.

[0077] like Figure 3 The image shown is a data fusion system based on federated computing under resource-constrained conditions, provided in an embodiment of this application, comprising: Multiple user terminals N i (i=1,2,…,n), multiple agent training nodes G p(p=1,2,…,m), and training aggregation node C; each user terminal N i Each with at least one agent training node G p Communication connection, proxy training node G p It communicates with the training aggregation node C; User terminal N i This is used to generate inference upload data based on user-inputted inference input data, and then send the inference upload data to the target agent training node G. pp and receiving forwarded data from the target agent training node G pp The fused training data is used to update the local terminal model; Agent training node G p Used to respond to a message sent from the target user terminal N ii The system uploads inference data, trains the local agent training model based on the inference data to obtain local training data, uploads the local training data to the training aggregation node C, and, upon receiving the fused training data, forwards the fused training data to the target user terminal N. ii ; Training aggregation node C is used to receive data sent from multiple agent training nodes G. p The training data is processed, and fused training data is generated based on multiple training data sets. The fused training data is then distributed to each agent training node G. p .

[0078] In summary, in this embodiment, the user terminal first sends inference upload data reflecting the user's latest needs and environmental status to the target proxy training node. This allows the proxy training node to process the data as training material, ensuring the timeliness of data utilization and avoiding training lag caused by delayed uploads or post-processing, effectively enhancing the model's adaptability to dynamic environments. Furthermore, after receiving the inference upload data, the proxy training node performs local training based on the data and generates local training data, which is then uploaded to the training aggregation node. This not only alleviates the technical bottleneck of terminal devices being unable to perform local training due to limited computing power but also achieves effective communication between data and the model through the relay training mechanism of the proxy node. The coupling mechanism reduces the resource burden on terminals during training while improving the system's scalability and deployment flexibility. Then, after the training aggregation node receives training data from multiple proxy training nodes, it generates fused training data with global data attributes through data fusion. This fully integrates data features from different terminal environments, improving the model's generalization ability and overall accuracy, and reducing training bias caused by data silos or environmental differences. Finally, the proxy training nodes synchronize the model results from the fused training data to the corresponding user terminals, achieving closed-loop model iteration driven by inference data. This allows the terminal model to quickly respond to changes in user behavior, further improving inference accuracy and system response efficiency while ensuring inference performance. Therefore, based on the method of this application embodiment, by constructing a hierarchical structure of "user terminal—proxy training node—training aggregation node," it achieves efficient utilization of inference data and continuous model optimization under resource-constrained conditions. This solves the problem of model update lag caused by the inability of terminals to train locally and the waste of inference data timeliness in existing centralized training architectures, improving the adaptability, training efficiency, and deployment flexibility of the "Magic Inference" system.

[0079] like Figure 4 The figure shows a deployment method for a data fusion system based on federated computing under resource-constrained conditions, provided by an embodiment of this application.

[0080] Specifically, the following steps are included: Step 301: Using the user terminal-agent training node relationship requirement of the data fusion system based on federated computing under resource-constrained conditions in this application embodiment as the first constraint, the number of agent training nodes as the second constraint, and the agent training node-training aggregation node relationship requirement as the third constraint, and taking minimizing the resource consumption of the data fusion system based on federated computing under resource-constrained conditions as the optimization objective, an optimization function for the data fusion system based on federated computing under resource-constrained conditions is established.

[0081] In some embodiments of this application, to achieve efficient deployment of the federated computing system under resource-constrained conditions, an optimization function is established with the connection requirements between user terminals and agent training nodes as the first constraint, the number of agent training nodes as the second constraint, and the connection requirements between agent training nodes and training aggregation nodes as the third constraint. The optimization objective is to minimize system resource consumption. Specifically, based on parameters such as the model requirements of terminal devices, the container capacity of agent servers, and image storage limitations, an optimization function is constructed that includes variables such as device-model association, container deployment, and image allocation, thereby forming a solvable deployment model. The optimization function describes how the system can complete model container deployment and node allocation with minimal resource consumption while satisfying connection requirements. In this way, the system can achieve mathematical modeling of deployment schemes in complex topologies with multiple terminals, multiple agent nodes, and multiple aggregation nodes, providing a theoretical basis for solving subsequent deployment parameters and system implementation.

[0082] In a specific example, researchers deployed a federated learning architecture with multiple camera terminals in a smart security system. Each camera terminal required a different image recognition model, deployed across multiple home gateway proxy nodes. The system constructed an optimization function based on the model requirements of each terminal, the container capacity of the proxy nodes, and their connection to the central server, and introduced correlation constraints for modeling. This yielded mathematical expressions that could be used to solve for subsequent deployment parameters, providing theoretical support for efficient model allocation and container deployment in resource-constrained home network environments.

[0083] To further clarify the technical solution of this application, a specific optimization model is given below to illustrate the optimization logic of this solution under resource-constrained conditions. It should be noted that this mathematical model is merely an exemplary implementation and does not constitute a limitation on the technical solution of this application. Those skilled in the art can adjust and extend the model structure, variable settings, or optimization objectives according to actual deployment needs, and such adjustments should still be considered within the protection scope of this application. In some embodiments of this application, considering the large number of user terminals and the situation where multiple devices share the same model or a single device corresponds to multiple models, the system designs a containerized deployment mechanism based on federated computing. In this mechanism, each agent training node acts as an image pool to hold multiple model images and runs the models required by each device through container instances. Each container is used to host a running instance of a model. The total number of containers under the agent training node is set as a capacity constraint Q, and the total number of images that can be supported is set as D. Containers can load the corresponding model images from the image pool according to device needs to achieve dynamic distribution and execution of models.

[0084] During deployment, the system constructs an optimization function and solves the mapping relationship between proxy training nodes and user terminals based on the number of user terminals (n), the number of models (k), the number of proxy training nodes (m), and the container capacity constraints of each proxy training node. Through this mapping relationship, the system rationally allocates the model images required by each user terminal to the proxy training nodes and completes container deployment, thereby achieving efficient model proxy and inference support under resource-constrained conditions. The following model can then be established: User terminal set: Model set: Set of agent training nodes: ; And establish a user terminal-model association matrix. , in Indicates equipment Model needed Otherwise, it is 0.

[0085] And establish resource restrictions for agent training nodes as follows: Each agent training node Maximum number of containers: (Total number of containers is constrained) ) Agent training node The mirror set and: .

[0086] Indicates user terminal model Deployed on the agent training node The value is 0 if it is in the container, otherwise it is 0.

[0087] Indicates agent training node Existential Model The mirror image of the image; otherwise, the value is 0.

[0088] So, what is the minimum objective function for container deployment? for: This represents the total image storage and container resource consumption.

[0089] At this point, the three main constraints of the federated learning container deployment problem are as follows: First constraint (user terminal-agent training node relationship requirement): This means that the model required for each device must be deployed in at least one container on a proxy server; Second constraint (number of proxy training nodes): This indicates that the total number of containers on the proxy server does not exceed [a certain threshold]. .

[0090] Third constraint (relationship requirement between proxy training node and training aggregation node): This means that the corresponding container can only be deployed if the model image exists on the proxy server.

[0091] Step 302: Solve the optimization function to obtain the deployment parameters for the data fusion system under resource-constrained conditions based on federated computing, and deploy the data fusion system under resource-constrained conditions based on federated computing according to the deployment parameters.

[0092] In some embodiments of this application, to determine the specific deployment scheme of the federated computing system under resource-constrained conditions based on the established optimization function, the optimization function is solved to obtain the deployment parameters for the system, and the data fusion system based on federated computing under resource-constrained conditions is deployed according to the deployment parameters. Specifically, after completing the optimization function modeling, the Lagrange relaxation method can be used to solve the objective function, iteratively updating the Lagrange multipliers, decomposing it into image deployment sub-problems and container deployment sub-problems, and finally obtaining the deployment parameters for image allocation, container allocation, and device mapping. The deployment parameters refer to the structured configuration results used to guide the actual deployment of the system, including the binding relationship between proxy servers and model images, the container quantity allocation scheme, and the mapping relationship between terminal devices and proxy nodes. In this way, the system can complete the automated deployment of the federated computing architecture while meeting resource constraints and connectivity requirements, improving the system's scalability and deployment efficiency.

[0093] In a specific example, when deploying a federated learning system for a smart park, researchers faced complex deployment requirements involving multiple camera terminals, multiple image recognition models, and multiple edge proxy servers. After modeling the optimization function, the system uses the subgradient method to iteratively update the Lagrange multipliers, ultimately solving for the set of model images, the number of containers, and the mapping relationship with terminal devices that should be deployed on each proxy server. Based on the solution results, the system automates the deployment of the federated computing system, enabling each terminal device to efficiently access the federated learning process in a resource-constrained edge environment, meeting the collaborative needs of model distribution, training relay, and inference updates.

[0094] Optionally, in order to solve the optimization function, step 302 includes the following sub-steps: Sub-step 3021: Determine the dual optimization function of the optimization function using the Lagrange relaxation method.

[0095] In some embodiments of this application, to transform the primal optimization problem containing multiple constraints into a decomposable form for subsequent solution, a dual optimization function can be determined using the Lagrange relaxation method. Specifically, for the device-model requirement constraints, proxy server container capacity constraints, and image-container association constraints in the primal optimization function, corresponding Lagrange multipliers can be introduced to transform the constraint terms in the primal problem into multiplier terms and add them to the objective function, thereby constructing the dual optimization function. The Lagrange relaxation method is a mathematical method for handling constrained optimization problems. Its core idea is to transform the primal problem into an unconstrained dual problem by introducing multiplier variables, facilitating decomposition and iteration in complex deployment scenarios. In this way, the system can transform the primal deployment optimization problem into a clearly structured, iteratively solvable mathematical expression, providing a basic function structure for subsequent subgradient iterations.

[0096] In a specific example, when deploying a federated learning system for smart buildings, researchers needed to allocate models between terminal devices on multiple floors and multiple edge agent servers. Based on device requirements, server capacity, and mirroring distribution rules, the system constructed a primal optimization function. In this step, the Lagrange relaxation method was used to introduce multiplier variables, transforming the constraint terms into a dual optimization function. Ultimately, the system obtained a mathematical expression that could be used for subsequent iterative solutions, making the deployment problem decomposable and controllable, laying the foundation for efficient calculation of subsequent deployment parameters.

[0097] Sub-step 3022: Solve the dual optimization function using the subgradient method to iteratively update the Lagrange multipliers of the dual optimization function, and determine the solution of the optimization function when the Lagrange multipliers reach the preset iteration termination condition.

[0098] In some embodiments of this application, to efficiently solve the dual optimization function in resource-constrained deployment scenarios, the subgradient method can be used to solve the dual optimization function. This involves iteratively updating the Lagrange multipliers of the dual optimization function and determining the solution to the optimization function when the Lagrange multipliers reach a preset iteration termination condition. Specifically, based on the constructed dual optimization function, the system initializes the Lagrange multipliers and uses the subgradient method to update the multipliers round by round. In each iteration, the subgradient direction is calculated based on the current multiplier value and adjusted with a preset step size until the termination condition is met. In this way, the system can gradually approach the optimal deployment parameters under complex constraints and ultimately obtain an optimization function solution that satisfies resource constraints and connectivity requirements.

[0099] In a specific example, when deploying a federated learning system for a smart campus, the system initializes Lagrange multipliers based on a previously constructed dual optimization function and iteratively updates them using the subgradient method. In each iteration, the system calculates the subgradient direction based on the current multiplier value and adjusts the multipliers with decreasing step sizes until the iteration termination condition is met. In this way, the system ultimately obtains deployment parameters that satisfy device requirements, container capacity, and image association constraints, providing an optimized deployment scheme for model allocation between various terminal devices and proxy servers within the campus.

[0100] Based on the constraints given after step 301, the following three Lagrange multipliers can be introduced: Will As the multiplier of the first constraint condition, and will The multiplier as the first constraint, and the .... As the multiplier of the first constraint, the constraint can be transformed into a Lagrange term and added to the objective function: , By recombining the various elements, we get: , At this point, considering that the key advantage of the Lagrange relaxation method lies in decomposing the original problem into multiple independent subproblems, the Lagrange dual problem using the original problem becomes: Then we can consider the following issues separately: Image deployment sub-problem: Determine which images should be deployed on which agent training nodes; Container deployment sub-problem: Determine which agent training node's container the model for each user terminal should be deployed in.

[0101] Furthermore, considering the Lagrange dual function Typically, the problem is piecewise linear (non-differentiable), so the subgradient method can be used to update the Lagrange multipliers to obtain the final iterative objective, thereby solving the optimization problem. , , , in It is the step size, usually taken as... t represents the number of learning iterations.

[0102] refer to Figure 5 It illustrates a data fusion device 41 based on federated computing under resource-constrained conditions, provided in an embodiment of this application, applied to a local user terminal, including: The input upload module 411 is used to generate inference upload data based on the inference input data input by the user, and send the inference upload data to the target agent training node; The model download module 412 is used to receive fused training data forwarded from the target agent training node and update the local terminal model based on the fused training data.

[0103] Optionally, the data fusion apparatus 41 based on federated computing under resource-constrained conditions further includes: The inference execution module is used to input inference input data into the updated local terminal model in order to execute inference tasks.

[0104] Optionally, the input upload module 411 includes: The input upload submodule is used to determine the inference input data and / or the summary data of the inference input data entered by the user as the inference upload data.

[0105] refer to Figure 6 It illustrates a data fusion device 42 based on federated computing under resource-constrained conditions, provided in an embodiment of this application, applied to a local agent training node, including: The local training module 421 is used to train the local agent training model based on the inference upload data sent from the target user terminal in response to the inference upload data, so as to obtain local training data. The training upload module 422 is used to upload local training data to the training aggregation node. After receiving training data sent from multiple proxy training nodes, the training aggregation node generates fused training data based on the multiple training data and distributes the fused training data to each proxy training node. The model forwarding module 423 is used to forward the fused training data to the target user terminal upon receiving the fused training data.

[0106] Optionally, the local training module 421 includes: The local training submodule is used to perform subgradient iteration on the loss function of the local agent training model based on the inference uploaded data to obtain the trained local agent training model. The training extraction submodule is used to extract local training data from the trained local agent training model.

[0107] refer to Figure 7 It illustrates a data fusion device 43 based on federated computing under resource-constrained conditions, provided in an embodiment of this application, applied to training aggregation nodes, including: The training fusion module 431 is used to receive training data sent from multiple proxy training nodes and generate fused training data based on the multiple training data. The model distribution module 432 is used to distribute the fused training data to each agent training node; the agent training node that receives the fused training data is used to forward the fused training data to the target user terminal; the target user terminal is the user terminal that sends the target inference upload data to the agent training node; the target inference upload data is used to generate the training data sent from the agent training node.

[0108] Optionally, the training fusion module 431 includes: The training fusion submodule is used to aggregate multiple training data to obtain fused training data.

[0109] refer to Figure 8 It illustrates a deployment apparatus 40 for a resource-constrained data fusion system based on federated computing, provided in an embodiment of this application, comprising: The modeling module 401 is used to establish an optimization function for the data fusion system under resource-constrained conditions based on federated computing, with the user terminal-agent training node relationship requirement as the first constraint, the number of agent training nodes as the second constraint, and the agent training node-training aggregation node relationship requirement as the third constraint, and with the optimization objective of minimizing the resource consumption of the data fusion system under resource-constrained conditions based on federated computing. The configuration solution module 402 is used to solve the optimization function to obtain the deployment parameters of the data fusion system under resource-constrained conditions based on federated computing, and to deploy the data fusion system under resource-constrained conditions based on federated computing according to the deployment parameters.

[0110] Optionally, the solver configuration module 402 includes: The dual modeling submodule is used to determine the dual optimization function of the optimization function based on the Lagrange relaxation method. The solution submodule is used to solve the dual optimization function using the subgradient method, to iteratively update the Lagrange multipliers of the dual optimization function, and to determine the solution of the optimization function when the Lagrange multipliers reach the preset iteration termination condition.

[0111] In summary, in this embodiment, the user terminal first sends inference upload data reflecting the user's latest needs and environmental status to the target proxy training node. This allows the proxy training node to process the data as training material, ensuring the timeliness of data utilization and avoiding training lag caused by delayed uploads or post-processing, effectively enhancing the model's adaptability to dynamic environments. Furthermore, after receiving the inference upload data, the proxy training node performs local training based on the data and generates local training data, which is then uploaded to the training aggregation node. This not only alleviates the technical bottleneck of terminal devices being unable to perform local training due to limited computing power but also achieves effective communication between data and the model through the relay training mechanism of the proxy node. The coupling mechanism reduces the resource burden on terminals during training while improving the system's scalability and deployment flexibility. Then, after the training aggregation node receives training data from multiple proxy training nodes, it generates fused training data with global data attributes through data fusion. This fully integrates data features from different terminal environments, improving the model's generalization ability and overall accuracy, and reducing training bias caused by data silos or environmental differences. Finally, the proxy training nodes synchronize the model results from the fused training data to the corresponding user terminals, achieving closed-loop model iteration driven by inference data. This allows the terminal model to quickly respond to changes in user behavior, further improving inference accuracy and system response efficiency while ensuring inference performance. Therefore, based on the method of this application embodiment, by constructing a hierarchical structure of "user terminal—proxy training node—training aggregation node," it achieves efficient utilization of inference data and continuous model optimization under resource-constrained conditions. This solves the problem of model update lag caused by the inability of terminals to train locally and the waste of inference data timeliness in existing centralized training architectures, improving the adaptability, training efficiency, and deployment flexibility of the "Magic Inference" system.

[0112] Reference Figure 9 The electronic device 500 may include one or more of the following components: processing component 502, memory 504, power supply component 506, multimedia component 508, audio component 510, input / output (I / O) interface 512, sensor component 514, and communication component 516.

[0113] Processing component 502 typically controls the overall operation of electronic device 500, such as operations associated with display, telephone calls, data communication, camera operation, and recording. Processing component 502 may include one or more processors 520 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 502 may include one or more modules to facilitate interaction between processing component 502 and other components. For example, processing component 502 may include a multimedia module to facilitate interaction between multimedia component 508 and processing component 502.

[0114] Memory 504 is used to store various types of data to support the operation of electronic device 500. Examples of this data include instructions for any application or method operating on electronic device 500, contact data, phonebook data, messages, pictures, multimedia, etc. Memory 504 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0115] Power supply component 506 provides power to various components of electronic device 500. Power supply component 506 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic device 500.

[0116] Multimedia component 508 includes an interface that provides an output interface between electronic device 500 and user. In some embodiments, the interface may include a liquid crystal display (LCD) and a touch panel (TP). If the interface includes a touch panel, the interface may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may not only sense the boundaries of touch or swipe actions but also detect the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 508 includes a front-facing camera and / or a rear-facing camera. When electronic device 500 is in an operating mode, such as shooting mode or multimedia mode, the front-facing camera and / or rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0117] Audio component 510 is used to output and / or input audio signals. For example, audio component 510 includes a microphone (MIC) used to receive external audio signals when electronic device 500 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 504 or transmitted via communication component 516. In some embodiments, audio component 510 also includes a speaker for outputting audio signals.

[0118] Input / output (I / O) interface 512 provides an interface between processing component 502 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0119] Sensor assembly 514 includes one or more sensors for providing state assessments of various aspects of electronic device 500. For example, sensor assembly 514 may detect the on / off state of electronic device 500, the relative positioning of components such as the display and keypad of electronic device 500, changes in position of electronic device 500 or a component of electronic device 500, the presence or absence of user contact with electronic device 500, orientation or acceleration / deceleration of electronic device 500, and temperature changes of electronic device 500. Sensor assembly 514 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 514 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 514 may also include an accelerometer, gyroscope, magnetometer, pressure sensor, or temperature sensor.

[0120] Communication component 516 facilitates wired or wireless communication between electronic device 500 and other devices. Electronic device 500 can access wireless networks based on communication standards, such as WiFi, carrier networks (such as 2G, 3G, 4G, or 5G), or combinations thereof. In one exemplary embodiment, communication component 516 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 516 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0121] In an exemplary embodiment, the electronic device 500 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to implement the methods provided in the embodiments of this application.

[0122] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 504 including instructions, which can be executed by a processor 520 of an electronic device 500 to perform the above-described method. For example, the non-transitory storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0123] In an exemplary embodiment, the electronic device 500 may also be provided as a server, including a processing component 502, which further includes one or more processors 520, and memory resources represented by memory 504 for storing instructions, such as applications, that can be executed by the processing component 502. The applications stored in memory 504 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 502 is configured to execute instructions to perform the methods provided in the embodiments of this application.

[0124] Electronic device 500 may also include a power supply component 506 configured to perform power management of electronic device 500, a wired or wireless communication component 516 configured to connect electronic device 500 to a network, and an input / output (I / O) interface 512. Electronic device 500 may operate on an operating system stored in memory 504, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, or similar.

[0125] This application also provides a computer-readable storage medium storing a computer program. When executed by a processor, this computer program implements the various processes of the above-described embodiments of the data fusion method or deployment method of the data fusion system under resource-constrained conditions based on federated computing, and achieves the same technical effects. To avoid repetition, it will not be described again here. The computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0126] It should be noted that, for the sake of simplicity, the method embodiments of this application are all described as a series of actions. However, those skilled in the art should understand that the embodiments of this application are not limited to the described order of actions, because according to the embodiments of this application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily essential to the embodiments of this application.

[0127] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the application disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the claims below.

[0128] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the claimed rights.

[0129] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0130] It will be readily apparent to those skilled in the art that any combination of the above embodiments is feasible. Therefore, any combination of the above embodiments is an implementation scheme of this application. However, due to space limitations, this specification will not describe them in detail here.

[0131] The data fusion method or deployment method for resource-constrained data fusion systems based on federated computing provided herein is not inherently related to any particular computer, virtual system, or other device. Various general-purpose systems can also be used in conjunction with the teachings herein. Based on the above description, the required structure for constructing a system with the scheme of this application is obvious. Furthermore, this application is not directed to any particular programming language. It should be understood that the content of this application described herein can be implemented using various programming languages, and the above description of specific languages ​​is for the purpose of disclosing the best mode of implementation of this application.

[0132] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of this application may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0133] Similarly, it should be understood that, for the sake of brevity and to aid in understanding one or more of the various aspects of this application, in the above description of exemplary embodiments of this application, various features of this application are sometimes grouped together into a single embodiment, figure, or description thereof. However, this method of disclosure should not be construed as reflecting an intention that the claimed application contains more features than expressly recited in each of the claimed claims. Rather, as reflected in the claimed claims, the application aspects comprise fewer than all features of a single previously disclosed embodiment. Therefore, the specific implementation followed by the claimed claims can thus be explicitly incorporated into that specific implementation, wherein each claimed claim itself is a separate embodiment of this application.

[0134] Those skilled in the art will understand that modules in the device of the embodiments can be adaptively changed and placed in one or more devices different from that embodiment. Modules, units, or components in the embodiments can be combined into a single module, unit, or component, and further, they can be divided into multiple sub-modules, sub-units, or sub-components. Except where at least some of such features and / or processes or units are mutually exclusive, any combination of all features disclosed in this application and all processes or units of any method or device so disclosed can be employed. Unless expressly stated otherwise, each feature disclosed in this application may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.

[0135] Furthermore, those skilled in the art will understand that although some embodiments described herein include certain features but not others included in other embodiments, combinations of features from different embodiments are intended to be within the scope of this application and form different embodiments. For example, any one of the embodiments in the claimed claims can be used in any combination.

[0136] The various component embodiments of this application can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some or all of the components in the deployment method of the resource-constrained data fusion method or the resource-constrained data fusion system based on federated computing according to the embodiments of this application. This application can also be implemented as a device or apparatus program (e.g., a computer program and computer program product) for performing part or all of the methods described herein. Such an implementation of this application can be stored on a computer-readable medium, or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.

[0137] In another embodiment of the present invention, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute the data fusion method based on federated computing under resource-constrained conditions or the deployment method of the data fusion system based on federated computing under resource-constrained conditions according to the embodiments of the present application.

[0138] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0139] It should be noted that the above embodiments are illustrative of this application and not restrictive, and that those skilled in the art can devise alternative embodiments without departing from the scope of the claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. This application can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the claimed unit claims that enumerate several means, several of these means may be embodied by the same hardware item. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.

[0140] It should be noted that, for the sake of simplicity, the method embodiments of this application are all described as a series of actions. However, those skilled in the art should understand that the embodiments of this application are not limited to the described order of actions, because according to the embodiments of this application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily essential to the embodiments of this application.

[0141] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for embodiments of systems or devices, since they are basically similar to the method embodiments, the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0142] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A data fusion method based on federated computing under resource-constrained conditions, characterized in that, Applied to local user terminals, including: Inference upload data is generated based on the inference input data input by the user, and the inference upload data is sent to the target agent training node; Receive fused training data forwarded from the target agent training node, and update the local terminal model based on the fused training data.

2. A data fusion method based on federated computing under resource-constrained conditions, characterized in that, Applied to local agent training nodes, including: In response to inference upload data sent from the target user terminal, the local agent training model is trained based on the inference upload data to obtain local training data; The local training data is uploaded to the training aggregation node; after receiving the training data sent from multiple proxy training nodes, the training aggregation node generates fused training data based on the multiple training data and distributes the fused training data to each proxy training node. Upon receiving the fused training data, the fused training data is forwarded to the target user terminal.

3. The data fusion method based on federated computing under resource-constrained conditions as described in claim 2, characterized in that, The step of training the local agent training model based on the inference uploaded data to obtain local training data includes: Based on the inference uploaded data, the loss function of the local agent training model is subjected to subgradient iteration to obtain the trained local agent training model. The local training data is extracted from the trained local agent training model.

4. A data fusion method based on federated computing under resource-constrained conditions, characterized in that, Applied to training aggregation nodes, including: Receive training data sent from multiple proxy training nodes, and generate fused training data based on the multiple training data; The fused training data is distributed to each proxy training node; the proxy training node receiving the fused training data is used to forward the fused training data to the target user terminal; the target user terminal is the user terminal that sends target inference upload data to the proxy training node; the target inference upload data is used to generate training data sent from the proxy training node.

5. The data fusion method based on federated computing under resource-constrained conditions as described in claim 4, characterized in that, The step of generating fused training data based on multiple training data includes: The multiple training data sets are aggregated to obtain the fused training data.

6. A data fusion system based on federated computing under resource-constrained conditions, characterized in that, include: Multiple user terminals, multiple agent training nodes, and training aggregation nodes; Each user terminal is communicatively connected to at least one of the proxy training nodes, and the proxy training nodes are communicatively connected to the training aggregation node. The user terminal is used to generate inference upload data based on inference input data input by the user, send the inference upload data to the target agent training node, receive fused training data forwarded from the target agent training node, and update the local terminal model according to the fused training data. The proxy training node is used to respond to inference upload data sent from the target user terminal, train the local proxy training model based on the inference upload data to obtain local training data, upload the local training data to the training aggregation node, and forward the fused training data to the target user terminal when the fused training data is received. The training aggregation node is used to receive training data sent from multiple proxy training nodes, generate fused training data based on the multiple training data, and distribute the fused training data to each proxy training node.

7. A deployment method for a data fusion system under resource-constrained conditions based on federated computing, characterized in that, include: Using the user terminal-agent training node relationship requirement of the data fusion system based on federated computing under resource-constrained conditions as described in claim 7 as the first constraint, the number of agent training nodes as the second constraint, and the agent training node-training aggregation node relationship requirement as the third constraint, and taking minimizing the resource consumption of the data fusion system based on federated computing under resource-constrained conditions as the optimization objective, an optimization function for the data fusion system based on federated computing under resource-constrained conditions is established. Solve the optimization function to obtain the deployment parameters for the resource-constrained data fusion system based on federated computing, and deploy the resource-constrained data fusion system based on federated computing according to the deployment parameters.

8. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium, which, when executed by a processor, implements the method as described in any one of claims 1 to 5 or as described in claim 7.

9. An electronic device, characterized in that, It includes a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the method as claimed in any one of claims 1 to 5 or as claimed in claim 7.

10. A computer program product, characterized in that, The computer program product stores a computer program that, when executed by a processor, implements the steps of the method as described in any one of claims 1 to 5 or as described in claim 7.

Images