A Fragmented Inference Method and Apparatus Based on Dynamic Bandwidth Awareness in a Federated Model

Abstract

This invention discloses a method and apparatus for federated model sharding inference based on dynamic bandwidth awareness, belonging to the field of distributed machine learning technology, and mainly solving the problem of low sharding inference efficiency in network bandwidth fluctuation environments. The method mainly includes: predicting network state data based on a lightweight bandwidth prediction model to obtain bandwidth prediction values; optimizing the model sharding size using the bandwidth prediction value as a network state constraint to obtain the expected model sharding size, and constructing a multi-dimensional state vector based on task characteristics, network state data, bandwidth prediction values, and device state data; determining a model sharding acquisition strategy from preset model sharding strategies with different sharding granularities based on the expected model sharding size and the multi-dimensional state vector, using a sharding decision model; determining at least one source node from the model sharding interaction network based on the model sharding acquisition strategy, and performing inference tasks based on the model shards received from each source node. This method is mainly used for model sharding inference.

Description

Technical Field

[0001] This invention relates to the field of distributed machine learning technology, and in particular to a method and apparatus for fragmented inference of a federated model based on dynamic bandwidth awareness. Background Technology

[0002] With the explosive growth of IoT devices, the amount of edge data is increasing rapidly, and traditional cloud computing's centralized data processing methods are facing high latency and bandwidth pressure. Federated model sharding, as an effective distributed inference technology, is widely used in the "cloud-edge-device" three-tier architecture where cloud computing resources are limited. By dividing a deep learning model into multiple sub-models and deploying them on different edge nodes to collaboratively execute inference, the computing and storage limitations of a single device can be overcome.

[0003] Existing federated learning uses a fixed sharding strategy, which does not take into account the impact of real-time changes in network conditions on model transmission between different inference nodes, as well as the device status of different inference nodes. When faced with significant fluctuations in network bandwidth between edge nodes, the sharding strategy is prone to mismatch with network status, resulting in blockage and delay in model shard transmission. As a result, the inference task execution node cannot obtain the model shards in time and execute the inference task, thus affecting the overall inference efficiency. Summary of the Invention

[0004] In view of this, the present invention provides a method and apparatus for federated model fragmentation inference based on dynamic bandwidth awareness. The main purpose is to solve the problem that existing fragmentation inference methods cannot adapt to the network state when the network bandwidth fluctuates significantly, resulting in low overall inference efficiency.

[0005] According to one aspect of the present invention, a method for fragmented inference of a federated model based on dynamic bandwidth awareness is provided, comprising: In response to an inference task request, the system acquires task characteristics, device status data, and network status data within a preset historical time period. Based on a lightweight bandwidth prediction model, the system performs prediction processing on the network status data to obtain a bandwidth prediction value. Using the bandwidth prediction value as a network state constraint, the model fragment size is optimized to obtain the expected model fragment size, and a multi-dimensional state vector is constructed based on the task characteristics, network state data, the bandwidth prediction value, and the device state data. Based on the expected model shard size and the multidimensional state vector, a model shard acquisition strategy is determined from preset model sharding strategies with different sharding granularities using a sharding decision model. Based on the model fragment acquisition strategy, at least one source node and the corresponding model fragments to be transmitted for each source node are determined from the model fragment interaction network. A transmission request for the corresponding model fragment to be transmitted is sent to the source node, and an inference task is performed based on the model fragments received from each source node.

[0006] Furthermore, before performing prediction processing on the network state data based on the lightweight bandwidth prediction model to obtain the bandwidth prediction value, the method further includes: The initial lightweight bandwidth prediction model is received from the regional data center, wherein the initial lightweight bandwidth prediction model is obtained by pruning a pre-trained bidirectional long short memory neural network model. Based on real-time network status data, the model parameters of the initial lightweight bandwidth prediction model are optimized using sliding window incremental learning to obtain the change in model parameters after single-node optimization. The model parameter changes are uploaded to the regional data center so that the regional data center can perform global model aggregation based on the model parameter changes to obtain a lightweight bandwidth prediction model for distribution to each node. Receive the lightweight bandwidth prediction model issued by the regional data center.

[0007] Further, using the bandwidth prediction value as a network state constraint, the model fragment size is optimized to obtain the expected model fragment size, including: The inference model corresponding to the inference task is virtually fragmented to obtain multiple virtual model fragments, and the expected transmission bandwidth and importance parameters corresponding to each virtual model fragment are determined. Substituting the expected transmission bandwidth and importance parameters into the bandwidth-delay joint optimization function, we obtain the objective function. With minimizing the result of the bandwidth-delay joint optimization function as the objective and the expected fragmentation size as the optimization variable, we optimize the objective function to obtain the expected model fragmentation size. The bandwidth-delay joint optimization function comprises the sum of two function terms: the first function term is the sum of the ratios of the importance parameters of all virtual model fragments to the expected transmission bandwidth, and the second function term is the expected transmission delay, wherein the expected transmission bandwidth is constrained by the bandwidth prediction value.

[0008] Further, based on the expected model shard size and the multidimensional state vector, a model shard acquisition strategy is determined from preset model sharding strategies with different sharding granularities using a sharding decision model, including: Using the expected model partition size as a prior value, the initial selection probability of each preset model partitioning strategy in the partitioning decision model is initialized to obtain the prior probability distribution. The preset model partitioning strategy includes at least one of the following: full model transmission strategy, hierarchical partitioning strategy, channel partitioning strategy, neuron partitioning strategy, and partitioning strategy based on importance parameter proportion. Based on the multidimensional state vector and the prior probability distribution, the evaluation value of each preset model sharding strategy is calculated based on the sharding decision model. The multidimensional state vector includes at least one of the following: bandwidth prediction value, real-time bandwidth, network latency, packet loss rate, terminal remaining battery power, terminal memory usage rate, terminal processor load, task urgency, number of consecutive uses of the model sharding strategy, historical average transmission time, and historical transmission success rate. The model sharding acquisition strategy is determined from the preset model sharding strategy based on the evaluation value.

[0009] Furthermore, before determining the model sharding acquisition strategy from preset model sharding strategies with different sharding granularities based on the expected model sharding size and the multidimensional state vector using the sharding decision model, the method further includes: Initialize the state policy value table, wherein the state policy value table includes the state dimension corresponding to the multi-dimensional state vector and the policy dimension corresponding to the preset model slicing policy; Based on the real-time multidimensional state vector, a target sharding strategy is selected and executed from the preset model sharding strategy using a greedy algorithm; Obtain the actual transmission time, actual energy consumption, and model accuracy impact factor after the target fragmentation strategy is executed; The actual transmission time, the actual energy consumption, and the model accuracy impact factor are substituted into the sharding strategy reward function to calculate the reward value, wherein the sharding strategy reward function includes a transmission efficiency term and an energy consumption term; Based on the real-time multidimensional state vector, the target sharding strategy, the reward value, and the multidimensional state vector after the target sharding strategy is executed, update the strategy utility values ​​corresponding to different state strategy combinations in the state strategy value table. Repeat the operations of selecting the target sharding strategy, calculating the reward value, and updating the state strategy value table until the strategy utility value converges.

[0010] Further, based on the model fragment acquisition strategy, at least one source node and the corresponding model fragments to be transmitted for each source node are determined from the model fragment interaction network, including: According to the granularity of the model fragment acquisition strategy, the inference model corresponding to the inference task is logically divided into at least one model fragment to be transmitted, and priority labels are configured for each model fragment to be transmitted according to the transmission priority determined by the model fragment acquisition strategy. In the model sharding interaction network, the list of currently available source nodes is obtained through the node discovery protocol, and the model sharding cache information of each node is queried. Based on the model sharding cache information, at least one node that caches the corresponding model shard to be transmitted is matched as the source node for the model shard to be transmitted, and the model shard to be transmitted is associated with the source node.

[0011] Furthermore, for any of the aforementioned source nodes, the transmission process of the model fragment to be transmitted includes: The sensitivity of the fragment to be transmitted is calculated based on the statistical distribution of the fragmentation parameters; Based on the aforementioned sensitivity and privacy budget, a mixed noise mechanism of Laplace noise and Gaussian noise is used to add noise to the fragments of the model to be transmitted; The noise-added model fragments to be transmitted are subjected to range constraints to obtain noisy model fragments, and the noisy model fragments are transmitted to the node that issued the transmission request for the model fragments to be transmitted.

[0012] According to another aspect of the present invention, a federated model fragmented inference apparatus based on dynamic bandwidth awareness is provided, comprising: The bandwidth prediction module is used to respond to inference task requests, acquire task characteristics, device status data, and network status data within a preset historical time period, and perform prediction processing on the network status data based on a lightweight bandwidth prediction model to obtain bandwidth prediction values. The module is used to optimize the model fragment size with the bandwidth prediction value as a network state constraint, obtain the expected model fragment size, and construct a multi-dimensional state vector based on the task characteristics, network state data, bandwidth prediction value and device state data. The decision module is used to determine the model sharding acquisition strategy from preset model sharding strategies with different sharding granularities based on the expected model sharding size and the multidimensional state vector, according to the sharding decision model. The inference module is used to determine at least one source node and the corresponding model fragments to be transmitted from the model fragment interaction network according to the model fragment acquisition strategy, send the corresponding model fragment to be transmitted to the source node, and perform inference tasks based on the model fragments received from each source node.

[0013] According to another aspect of the present invention, a storage medium is provided, wherein at least one executable instruction is stored therein, the executable instruction causing a processor to perform operations corresponding to the above-described dynamic bandwidth-aware federated model fragmented inference method.

[0014] According to another aspect of the present invention, a terminal is provided, comprising: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other through the communication bus; The memory is used to store at least one executable instruction that causes the processor to perform the operation corresponding to the above-described dynamic bandwidth-aware federated model fragmented inference method.

[0015] By employing the above-described technical solutions, the technical solutions provided by the embodiments of the present invention have at least the following advantages: This invention provides a method and apparatus for federated model sharding inference based on dynamic bandwidth awareness. In embodiments of this invention, in response to an inference task request, task features, device status data, and network status data within a preset historical time period are acquired. The network status data is then processed using a lightweight bandwidth prediction model to obtain a bandwidth prediction value. Using this bandwidth prediction value as a network status constraint, the model sharding size is optimized to obtain the expected model sharding size. A multi-dimensional state vector is constructed based on the task features, network status data, the bandwidth prediction value, and the device status data. Based on the expected model sharding size and the multi-dimensional state vector, a sharding decision model determines a model sharding acquisition strategy from preset model sharding strategies with different sharding granularities. Based on the model sharding acquisition strategy, at least one source node and corresponding model shards to be transmitted for each source node are identified from the model sharding interaction network. A transmission request for the corresponding model shard to be transmitted is sent to the source node, and an inference task is executed based on the model shards received from each source node. Compared to existing technologies that employ fixed strategies, this invention dynamically adjusts the fragment acquisition strategy based on real-time changes in network status, device status, and task characteristics. This effectively solves the problems of fragment mismatch and low transmission efficiency caused by fixed strategies failing to adapt to dynamic environmental changes. It significantly reduces invalid transmission and waiting time during inference task execution, while ensuring the timeliness of model fragment transmission and the continuity of task execution.

[0016] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention, it can be implemented according to the contents of the specification. Furthermore, in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0017] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 The flowchart of a fragmented inference method for a federated model based on dynamic bandwidth awareness provided by an embodiment of the present invention is shown. Figure 2 The flowchart of a lightweight bandwidth prediction model training method provided by an embodiment of the present invention is shown. Figure 3 This diagram illustrates a block diagram of a federated model fragmented inference device based on dynamic bandwidth awareness, provided by an embodiment of the present invention. Figure 4 A schematic diagram of the structure of a terminal provided in an embodiment of the present invention is shown. Detailed Implementation

[0018] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0019] To address the issue of low overall inference efficiency in existing fragmented inference methods under environments with significant network bandwidth fluctuations, this invention provides a fragmented inference method based on a dynamic bandwidth-aware federated model, such as... Figure 1 As shown, the method includes: 101. In response to the inference task request, acquire task characteristics, device status data, and network status data within a preset historical time period, and perform prediction processing on the network status data based on a lightweight bandwidth prediction model to obtain a bandwidth prediction value.

[0020] In this embodiment of the invention, the current execution entity is any node in the model sharding interaction network, i.e., a terminal device or lightweight server deployed at the network edge. This node plays a dual role as both a data producer and a consumer. When this node receives an inference task request, it acquires task characteristics such as the amount of data to be processed, the urgency of task processing, and the sensitivity of information content for the inference task to be executed, as well as device status data of the terminal device corresponding to the current node, such as remaining battery power, CPU load, and memory usage. In addition, to enable the model sharding strategy to adapt to the link quality between edge nodes, it also acquires network status data collected in real time by the current edge node within a preset historical period, such as bandwidth data collected in real time within 30 or 60 seconds prior to the current time. The bandwidth is predicted for a future period based on a lightweight bandwidth prediction model pre-deployed on the current edge node, resulting in a bandwidth prediction value. The lightweight bandwidth prediction model can be a bidirectional long short-term memory neural network model architecture, including 128 neurons at the beginning and end, and introducing an attention mechanism. After pruning and quantization processing, this model structure is a single-layer long short-term memory neural network with 16 hidden layer units, deployed on the edge node. Network status data is a set of link quality metrics between the current node and other peer nodes in the network, including real-time bandwidth, network latency, packet loss rate, etc.

[0021] In this invention, inference task requests can be triggered by preset time nodes, data anomalies, or data processing tasks; specific limitations are not imposed on these instances. In a practical application of an intelligent traffic management system in a large city, each intersection deploys intelligent cameras and edge servers with computing capabilities. These cameras and edge servers are connected via 5G and fiber optic networks, forming a city-wide edge computing network. The edge computing network employs a hierarchical deployment strategy, with intersection cameras serving as the most basic edge nodes, regional servers as aggregation nodes, and the city-level data center as the global aggregation node. The sharding strategy is dynamically adjusted based on actual network conditions. During nighttime network downtime, the system uses a full-model transmission strategy to fully utilize idle bandwidth resources. During daytime peak traffic periods, when bandwidth fluctuates significantly, the system automatically switches to a hierarchical sharding strategy to reduce the impact on the core network. In special circumstances such as extreme weather, a critical gradient transmission strategy is activated to ensure the normal operation of key functions.

[0022] 102. Using the bandwidth prediction value as a network state constraint, optimize the model fragment size to obtain the expected model fragment size, and construct a multi-dimensional state vector based on the task characteristics, network state data, the bandwidth prediction value, and the device state data.

[0023] In this embodiment of the invention, after completing network state perception, prior knowledge required for subsequent reinforcement learning decisions is generated based on the predicted bandwidth values ​​of the links between the current node and other nodes, and a complete decision state space is constructed. Specifically, a bandwidth-latency joint optimization function can be constructed by combining the mapping relationship between fragment size and transmission delay, using the current bandwidth prediction value as a network condition constraint. This optimization function uses mathematical analytical methods to solve for the theoretically optimal fragment size under the expected network state in the future, forming prior knowledge to guide subsequent decisions. Simultaneously, the node performs feature engineering processing on the collected task features, network state data, bandwidth prediction values, and device state data to extract multi-dimensional features, thereby constructing a multi-dimensional state vector that can be input to the reinforcement learning model.

[0024] 103. Based on the expected model shard size and the multidimensional state vector, a model shard acquisition strategy is determined from preset model sharding strategies with different sharding granularities using a sharding decision model.

[0025] In this embodiment of the invention, the constructed multidimensional state vector and the expected model slice size are jointly input into the slice decision model. This slice decision model can be a Q-learning reinforcement learning algorithm, where the Q-table predefines the state-policy value function mapping relationship. During the decision-making process, the expected model slice size provides a theoretically optimal slice size reference for the model's decision, reducing the decision search space. For example, a larger expected slice size tends to favor the full model transmission strategy, while a smaller expected slice size tends to favor the neuron slice strategy. The multidimensional state vector describes the overall state of the current network, device, and task, and as an index to the Q-table, it enables the decision to adapt to the current network and device states. By using the expected model slice size (referring to the theoretically optimal solution) and the multidimensional state vector describing the current environment state as dual inputs, the decision model can more efficiently select the optimal model slice acquisition strategy from the discrete policy space under the guidance of prior knowledge. Furthermore, the multidimensional state vector, which includes device state and bandwidth prediction values, guides the model to make slice strategies that are more adapted to the current network state and the actual performance of the device.

[0026] It should be noted that while the reinforcement learning-based decision-making mechanism described above can achieve refined policy selection in dynamic environments, in practical deployments, if faced with limited computing resources or high real-time requirements for decision-making, a simpler heuristic decision-making approach can be used as an alternative or supplement. As another simpler decision-making method, decisions can also be made based on the width prediction value in the multi-dimensional state vector, dividing the network state into three levels: high bandwidth (width prediction value greater than 50 Mbps), medium bandwidth (width prediction value between 10 and 50 Mbps), and low bandwidth (width prediction value less than 10 Mbps). Different fragmentation transmission strategies are adopted under different levels: under high bandwidth, complete model fragments including all parameters are transmitted to fully utilize bandwidth resources; under medium bandwidth, fragments selected based on importance are transmitted, such as only transmitting the top 30% of parameters with the highest absolute gradient values; under low bandwidth, only critical gradients are transmitted, such as the top 10% of parameters with the highest absolute gradient values ​​or parameters whose model update amount exceeds a threshold.

[0027] 104. Based on the model fragment acquisition strategy, determine at least one source node and the model fragments to be transmitted corresponding to each source node from the model fragment interaction network, send a transmission request for the corresponding model fragment to be transmitted to the source node, and perform inference tasks based on the model fragments received from each source node.

[0028] In this embodiment of the invention, the model fragment acquisition strategy can include various fragmentation strategies with different model fragmentation granularities, such as a full model transmission strategy without fragmentation, a layer-by-layer fragmentation strategy with the entire convolutional layer as the smallest transmission unit, a channel-by-channel fragmentation strategy with a subset of channels of the convolutional layer as the smallest transmission unit, and a neuron-by-neuron fragmentation strategy with a subset of neurons of the fully connected layer as the smallest transmission unit. The model fragmentation interaction network includes multiple nodes, each storing partial fragments of the model. That is, each model fragment to be transmitted is stored in more than one node in the model fragmentation interaction network. These nodes are interconnected through a dandelion-like topology, forming a decentralized fragmentation exchange network. While storing model fragments, each node also maintains locally observed network state data, including current bandwidth, latency, packet loss rate, and device status data, including remaining battery power, CPU load, and memory usage. It also periodically exchanges fragmentation index information with neighboring nodes, enabling each node to perceive the distribution of available fragments in the network. When any node initiates an inference task request, that node acts as the requesting node. It determines the set of candidate source nodes that have the target model shards based on the shard index stored locally, and selects the best one based on the real-time network measurement results of each candidate node. It then determines the final shard source node and its corresponding shard to be transmitted, thus achieving multi-source parallel acquisition. This ensures transmission efficiency while making full use of the model resources stored in the network.

[0029] After identifying the source nodes, the current node sends transmission requests for the corresponding model fragments to be transmitted to each source node, enabling the source nodes to transmit the corresponding model fragments to the current node. During the transmission process, different transmission priorities can be assigned to different fragments using a multi-stream concurrent approach to ensure that critical fragments are delivered first. After receiving the data, the node locally combines the fragments into a complete model and uses the combined model to perform inference tasks. Throughout this process, the node's sensitive raw data remains stationary; only the model fragments flow from the source nodes to the requesting nodes in the model fragment interaction network, completing model combination and inference locally. This protects data privacy and avoids the overhead of uploading and transmitting massive amounts of raw data.

[0030] In one embodiment of the present invention, for further illustration and limitation, such as Figure 2 As shown, before the step of using a lightweight bandwidth prediction model to predict the network state data and obtain the bandwidth prediction value, the method further includes: 201. Receive the initial lightweight bandwidth prediction model issued by the regional data center.

[0031] 202. Based on real-time network status data, optimize the model parameters of the initial lightweight bandwidth prediction model using sliding window incremental learning to obtain the change in model parameters after single-node optimization.

[0032] 203. Upload the model parameter changes to the regional data center so that the regional data center can perform global model aggregation based on the model parameter changes to obtain a lightweight bandwidth prediction model for distribution to each node.

[0033] 204. Receive the lightweight bandwidth prediction model issued by the regional data center.

[0034] In this embodiment of the invention, the regional data center acts as a global coordination node, uniformly distributing lightweight prediction models based on pruned and quantized algorithms to ensure that each edge node has executable baseline prediction capabilities. This regional data center can be a cloud data center or an intermediate data processing center connecting the cloud and edge nodes. The training process of the lightweight prediction model includes an offline training phase by the regional data center and an online training phase based on incremental learning by multiple edge nodes. At the beginning of model training, the regional data center uses a standard dataset to complete the initial training of the model, obtaining a pre-trained bandwidth prediction model. Then, the pre-trained bandwidth prediction model is pruned and quantized to obtain the initial lightweight bandwidth prediction model, which is then distributed to each edge node in the model sharding interaction network. That is, the initial lightweight bandwidth prediction model is obtained by pruning a pre-trained bidirectional long short-term memory neural network model.

[0035] At any given node, based on locally collected real-time network state data, a sliding window mechanism is used to maintain recent network state data. Incremental training is performed locally to obtain the single-node optimized model parameter changes that reflect local network characteristics, enabling the model to continuously adapt to the network changes in the node's environment. Furthermore, the node only uploads the parameter changes—not the complete model or original data—to the regional data center. The regional data center then uses a federated averaging algorithm to globally aggregate the parameter changes uploaded by each node, generating an updated lightweight bandwidth prediction model, which is then distributed to each node, completing one round of model optimization. Through this local node parameter optimization, the prediction model gradually adapts to the local network characteristics of each node while maintaining global generalization capabilities.

[0036] It should be noted that, to ensure the bandwidth prediction model can adapt to dynamic changes in the network environment, a sliding window optimization algorithm is used to dynamically adjust the prediction window size. This algorithm dynamically adjusts the sliding window size used for model training by calculating the relative error between the actual bandwidth and the bandwidth prediction value. For example, when the prediction error exceeds a threshold of 0.03, the window is smoothly expanded using a sigmoid function to obtain more historical information; when the error is within an acceptable range, the window is gradually reduced to decrease computational overhead. The window size is limited to between 5 and 15, with an initial value of 10 to match the 5G transmission time interval configuration.

[0037] In one embodiment of the present invention, for further explanation and limitation, the bandwidth prediction value is used as a network state constraint to optimize the model fragment size and obtain the expected model fragment size, including: The inference model corresponding to the inference task is virtually fragmented to obtain multiple virtual model fragments, and the expected transmission bandwidth and importance parameters corresponding to each virtual model fragment are determined. Substituting the expected transmission bandwidth and importance parameters into the bandwidth-delay joint optimization function yields the objective function. With minimizing the result of the bandwidth-delay joint optimization function as the objective and the expected fragment size as the optimization variable, the objective function is optimized to obtain the expected model fragment size.

[0038] In this embodiment of the invention, to determine the expected model fragment size, the inference model corresponding to the inference task is first virtually fragmented from a logical dimension, dividing the model into multiple virtual model fragments as optimization granularity units. Then, the expected transmission bandwidth of this virtual model fragment and the importance parameter used to characterize the influence of the model parameters contained in this fragment on the model accuracy, i.e., the L2 gradient norm of the model parameters, are substituted into the bandwidth-latency joint optimization function. The bandwidth-latency joint optimization function includes the sum of two function terms: the first function term is the sum of the ratios of the importance parameters of all virtual model fragments to the expected transmission bandwidth, and the second function term is the expected transmission latency. The expected transmission bandwidth is not the final allocated bandwidth, but rather a benchmark reference value in the optimization process. It can be estimated based on the bandwidth prediction values ​​in historical network states and multi-dimensional state vectors to estimate the bandwidth available in the current network, or it can be estimated based on bandwidth allocation records of similar fragments in historical transmissions. The first term ensures that important parameters are preferentially allocated to high-bandwidth resources, and the second term controls the impact of fragment size on transmission latency. The bandwidth-latency joint optimization function is specifically expressed as follows: ; in, This represents the size of the i-th virtual model slice, which serves as an optimization decision variable to determine the size of the i-th model slice. Let t represent the expected transmission bandwidth of the i-th virtual model fragment at time t. This expected transmission bandwidth is less than or equal to the bandwidth prediction value between the current node and the expected source node corresponding to the virtual model fragment. The expected source node can be the node with the largest bandwidth prediction value of the link with the current node and which stores the virtual model fragment. The L2 gradient norm of the model parameters represents the degree of influence of the i-th model slice on the model accuracy. express The expected transmission latency under the fragment size, where the transmission latency is extracted based on the mapping relationship between fragment size and transmission latency fitted based on historical data; The represents the balance coefficient, used to adjust the weights of the transmission efficiency term and the delay penalty term, and N represents the total number of model fragments. The above optimization function adjusts the size of each fragment. By minimizing the sum of the two terms, joint optimization of bandwidth utilization efficiency and transmission latency is achieved. A bandwidth prediction model provides forward-looking input to the optimization process, enabling the optimization function to perform fragmentation optimization based on forward-looking network conditions, thus avoiding short-sightedness caused by relying solely on the current instantaneous bandwidth.

[0039] In a specific application example, a three-tiered control architecture (macro-meta-micro) can be used to achieve end-to-end optimization from network fluctuation prediction to model fragmentation scheduling: The macro layer, acting as a trend prediction layer, predicts the bandwidth trend for the next 10 seconds based on a bidirectional long short-term memory neural network, providing forward-looking bandwidth constraints for determining the fragmentation transmission strategy. The meso layer, acting as a resource scheduling layer, dynamically divides the model into fragment combinations based on multi-dimensional state characteristics such as device computing power and network status, achieving load balancing and optimal resource utilization. The micro layer, acting as a transmission control layer, uses the QUIC protocol at the individual data connection level to implement fragment-level transmission priority scheduling. Under this layered architecture, the optimization process for the expected model fragment size does not need to be recalculated after each bandwidth prediction. That is, after completing the bandwidth prediction but before optimizing the fragment size, the current transmission quality can be evaluated based on a performance evaluation function. Only when the transmission quality does not meet the requirements is the expected model fragment size recalculated; otherwise, the previously calculated expected model fragment size can be used. The performance evaluation function is expressed as: ; in, Indicates transmission quality assessment parameters, Indicates the actual bandwidth used. This represents the bandwidth prediction value. Indicates the actual transmission delay. Indicates the target transmission delay. and This represents the weighting coefficient.

[0040] In one embodiment of the present invention, for further explanation and limitation, a model sharding acquisition strategy is determined from preset model sharding strategies with different sharding granularities based on a sharding decision model, including: Using the expected model partition size as a prior value, initialize the initial selection probability of each preset model partitioning strategy in the partitioning decision model to obtain the prior probability distribution; Based on the multidimensional state vector and the prior probability distribution, the evaluation value of each preset model partitioning strategy is calculated based on the partitioning decision model; The model sharding acquisition strategy is determined from the preset model sharding strategy based on the evaluation value.

[0041] In this embodiment of the invention, the expected model partition size is used as prior knowledge to initialize the initial selection probability of each preset model partitioning strategy in the partitioning decision model. The preset model partitioning strategies include at least one of the following: full model transmission strategy, hierarchical partitioning strategy, channel partitioning strategy, neuron partitioning strategy, and partitioning strategy based on importance parameters. The full model transmission strategy transmits the entire model without partitioning; the hierarchical partitioning strategy partitions the model based on network layers as the smallest unit; the channel partitioning strategy partitions the model based on subsets of channels within the network layers as the smallest unit; the neuron partitioning strategy partitions the model based on subsets of neurons as the smallest unit; and the partitioning strategy based on importance parameters selects a certain percentage of data from the parameter importance ranking for transmission, such as the top 50%, top 20%, etc. Furthermore, only key gradient parameters, i.e., parameters whose gradients exceed a threshold, can be transmitted.

[0042] In the fragmentation decision-making process, the fragmentation decision-making model first initializes the initial selection probabilities of each preset strategy using the expected fragmentation size as prior knowledge, forming a prior probability distribution. This distribution reflects the theoretical matching degree between different fragmentation granularity strategies and the theoretically optimal fragmentation size when there is no historical experience. For example, when the expected fragmentation is larger, the full-model transmission strategy obtains a higher initial probability, while when the expected fragmentation is smaller, the neuron-level fragmentation strategy obtains a higher initial probability. Subsequently, based on the constructed multidimensional state vector and the prior probability distribution, the evaluation value of each preset model fragmentation strategy is calculated based on the fragmentation decision-making model. The multidimensional state vector includes at least one of the following: bandwidth prediction value, real-time bandwidth, network latency, packet loss rate, terminal remaining battery power, terminal memory usage, terminal processor load, task urgency, number of consecutive uses of the model fragmentation strategy, historical average transmission time, and historical transmission success rate. The evaluation value is essentially a posterior probability estimate guided by prior knowledge. It's a comprehensive score of the actual applicability of each policy after observing the current environmental state. Prior probability provides the initial bias, while the multi-dimensional state vector scores the environmental state through a reinforcement learning Q-value function or a policy network. Both factors jointly determine the final evaluation value. Finally, the policy with the highest evaluation value is selected from a pre-defined policy set as the policy for model segmentation. This achieves a system where prior knowledge provides initial preferences, environmental state provides dynamic adjustments, and the evaluation value, combined with both, outputs the optimal decision.

[0043] In one embodiment of the present invention, for further explanation and limitation, before making a decision on the state vector based on the piecewise decision model to obtain the model piecewise acquisition strategy, the method further includes: Initialize the state policy value table; Based on the real-time multidimensional state vector, a target sharding strategy is selected and executed from the preset model sharding strategy using a greedy algorithm; Obtain the actual transmission time, actual energy consumption, and model accuracy impact factor after the target fragmentation strategy is executed; Substitute the actual transmission time, the actual energy consumption, and the model accuracy impact factor into the sharding strategy reward function to calculate the reward value; Based on the real-time multidimensional state vector, the target sharding strategy, the reward value, and the multidimensional state vector after execution, update the strategy utility values ​​corresponding to different state strategy combinations in the state strategy value table; Repeat the operations of selecting the target sharding strategy, calculating the reward value, and updating the state strategy value table until the strategy utility value converges.

[0044] In this embodiment of the invention, the training of the sharding decision model is a reinforcement learning optimization process combining offline feature extraction and online learning. At the beginning of training, a state-policy value table is initialized. The state dimension of this table is a multi-dimensional state vector, which is composed of the original features reduced to several key feature dimensions through principal component analysis. The key feature dimensions cover network metrics, such as smoothed bandwidth predictions, network jitter based on jitter buffer analysis, device status, such as power consumption calibrated by coulomb counting, CPU load isolated by the control group, task characteristics, such as transmission time predicted based on a regression model, sensitivity of predefined label system classification, and historical records, such as the number of consecutive policy uses, historical average transmission time, and historical transmission success rate. The policy dimension includes multiple preset model sharding strategies, each corresponding to a different model sharding granularity and different communication overhead. That is, the state dimension of the state-policy value table is a multi-dimensional state vector, and the policy dimension consists of multiple preset model sharding strategies. The smoothing formula for the bandwidth value is expressed as: BW_smooth(t) =α×BW(t) + (1-α)× BW_smooth(t-1); Here, the smoothing factor α can be set to 0.7, BW_smooth(t) represents the smoothed bandwidth value at the current time, BW(t) represents the original bandwidth measurement value at the current time, and BW_smooth(t-1) represents the smoothed bandwidth value at the previous time, which is used as a historical smoothed value in the current calculation.

[0045] Based on the currently collected multidimensional state vectors, an ε-greedy greedy strategy is used to select a target partitioning strategy from a preset strategy set. The optimal strategy in the current state (i.e., the strategy with the largest Q-value) is selected with a probability of 1-ε. Other strategies are randomly selected for exploration with a probability value of ε exploration rate to avoid getting trapped in local optima. After selecting the target partitioning strategy, the model is partitioned according to the corresponding granularity. For example, when selecting a hierarchical partitioning strategy, the residual network model containing 18 convolutional layers, quantized and compressed using a lightweight deep learning inference framework, is divided into multiple hierarchical partitions based on the convolutional layers. After execution, the actual transmission time, actual energy consumption, and model accuracy impact factors are collected, and these data are substituted into the partitioning strategy reward function to calculate the reward value. This reward function comprehensively considers key factors such as transmission efficiency and energy consumption, including a transmission efficiency term, an energy consumption term, and a model accuracy impact factor. The transmission efficiency term is determined based on the ratio of the baseline transmission time to the actual transmission time, and the energy consumption term is determined based on the percentage margin of the actual energy consumption relative to the maximum allowable energy consumption. The baseline transmission time is the time taken to transmit using the full model. The reward function for the sharding strategy can be specifically expressed as: ; in, This represents the reward value based on model accuracy. The weighting coefficient can be 0.6. The weighting coefficient can be 0.3. The weighting coefficient can be 0.1; Indicates the reference transmission time. Indicates the actual transmission time. Indicates actual energy consumption. Indicates the maximum allowable energy consumption. This represents the model accuracy impact factor, estimated using historical data. For example, it represents the change in model accuracy on the validation set after selecting different sharding strategies, establishing a mapping relationship between sharding strategies and accuracy loss. This serves as a priori reference for the model accuracy impact factor in subsequent reward calculations. It should be noted that the above sharding strategy reward function can be applied to federated learning training scenarios; however, in scenarios involving edge inference with appropriate incentive functions, the model accuracy impact factor can be replaced with a privacy-preserving term, resulting in a second reward function. ; in, This represents the reward value based on the hidden protection item. Anonymity is indicated by local differential privacy with a privacy budget of 0.8.

[0046] Furthermore, based on the real-time multidimensional state vector s, the selected target sharding policy a, the reward value r corresponding to the selected target sharding policy, and the new state vector observed after execution... Update the Q-table using the Q-learning update formula: ; in, The learning rate represents the degree to which new information affects the Q-value. This represents the discount factor, used to balance the weights of immediate rewards and future earnings. Indicates the assumption in state Below, multiple optional strategies The optimal return is determined by repeatedly performing the above operations of strategy selection, execution, reward calculation, and Q-table update, forming a continuous optimization loop. The training process employs an experience replay mechanism to store historical interaction samples, breaking data correlations through random sampling to improve learning stability; simultaneously, target network technology is introduced to periodically synchronize Q-network parameters, further stabilizing the training process. Model convergence is determined when the change in Q-value is less than a preset threshold or when strategy selection tends to stabilize over multiple consecutive evaluation rounds.

[0047] In one embodiment of the present invention, for further explanation and limitation, based on the model fragmentation acquisition strategy, at least one source node and the model fragments to be transmitted corresponding to each different source node are determined from the model fragmentation interaction network, including: According to the granularity of the model fragment acquisition strategy, the inference model corresponding to the inference task is logically divided into at least one model fragment to be transmitted, and priority labels are configured for each model fragment to be transmitted according to the transmission priority determined by the model fragment acquisition strategy. In the model sharding interaction network, the list of currently available source nodes is obtained through the node discovery protocol, and the model sharding cache information of each node is queried. Based on the model sharding cache information, at least one node that caches the corresponding model shard to be transmitted is matched as the source node for the model shard to be transmitted, and the model shard to be transmitted is associated with the source node.

[0048] In this embodiment of the invention, the inference model is logically divided into multiple model fragments to be transmitted according to the fragmentation granularity determined by the model fragmentation acquisition strategy. That is, the model is not actually divided here, but theoretically divided to determine the size and specific information of the model fragments to be acquired. Priority tags are configured for each fragment according to the transmission priority scheduling mechanism. This mechanism is based on the multi-stream characteristics of the QUIC protocol. For fragments with high task urgency or containing critical parameters, high-priority tags are assigned to ensure low-latency delivery; for non-critical fragments, ordinary priority tags are assigned, thus prioritizing the transmission of critical data under limited bandwidth conditions. Through the coordinated operation of priority tags and the multi-stream mechanism of the QUIC protocol, nodes can transmit multiple fragments in parallel within the same connection while dynamically adjusting the transmission order of each data stream.

[0049] The model sharding interaction network is constructed using a dandelion topology peer-to-peer (P2P) network structure. This dandelion topology mimics the propagation mechanism of dandelion seeds in nature, possessing characteristics such as self-organization, self-adaptation, and efficient propagation. In this topology, each edge node can act as a forwarding node for shards, constructing a dynamic, decentralized propagation network. When a node needs to obtain a model shard, it can download different shards in parallel from multiple neighboring nodes. Nodes obtain model shard cache information from each node through locally maintained shard indexes or by broadcasting query requests to neighboring nodes, including metadata such as the shard type, shard version, and storage location stored by each node. Because this topology does not rely on a central node, even if some nodes fail or leave the network, the system can still complete shard propagation through other paths. Furthermore, the process of adding new nodes is simple; they only need to establish connections with a few existing nodes to integrate into the entire network.

[0050] Based on the retrieved model fragment caching information, at least one node with a cached corresponding fragment is matched as a source node for each model fragment to be transmitted. Factors considered during the matching process include cache hit rate, link quality, and load balancing. The matching logic is as follows: nodes with cached target fragments are prioritized to avoid duplicate transmissions; real-time network measurement data of each candidate source node is used for selection, prioritizing nodes with good link quality; and leveraging the multi-path characteristics of the dandelion topology, different fragments are assigned to different source nodes to achieve parallel transmission and avoid single-point overload. After matching, each model fragment to be transmitted is associated with its corresponding source node, forming a fragment-source node mapping table, laying the foundation for subsequent transmission request sending and parallel acquisition. In the above source node matching process, the decentralized and multi-path parallel characteristics of the dandelion topology are fully utilized to maximize the efficiency of fragment acquisition.

[0051] In one embodiment of the present invention, for further explanation and limitation, the transmission process of the model fragment to be transmitted for any of the aforementioned source nodes includes: The sensitivity of the fragment to be transmitted is calculated based on the statistical distribution of the fragmentation parameters; Based on the aforementioned sensitivity and privacy budget, an adaptive noise is generated using a hybrid noise mechanism; The adaptive noise is added to the model fragment to be transmitted, and the value range of the fragment after adding noise is constrained to obtain the noisy model fragment; The noisy model fragment is transmitted to the node that issued the transmission request for the model fragment to be transmitted.

[0052] In this embodiment of the invention, the source node first calculates the sensitivity of the fragment parameters to be transmitted based on their statistical distribution. Sensitivity reflects the potential risk of privacy leakage from the fragment parameters. Fragments with a larger range of parameter values ​​and more drastic changes have higher sensitivity, indicating that a larger perturbation amplitude is needed to effectively protect privacy when adding noise. During the sensitivity calculation, the L2 norm reflecting the overall amplitude of the parameter vector in the model parameters is calculated, and the top 95% is taken, excluding 5% outliers to avoid excessive sensitivity due to outliers. Then, the sensitivity is calculated based on the model fragment parameters after removing outliers. The privacy budget can be a privacy budget value set by the receiving data owner or task initiator, or it can be obtained by querying a preset task privacy budget mapping relationship based on the inference task type corresponding to the model fragment to be transmitted. This embodiment of the invention does not impose specific limitations. The sensitivity calculation formula is expressed as: ; Where x and y are any two adjacent datasets in the local training data used to train the current model slice. This represents the parameter values ​​of the i-th model slice on dataset x. This represents the parameter values ​​of the i-th model slice in dataset y. This represents the L2 norm.

[0053] Based on the calculated shard sensitivity and privacy budget, the source node generates adaptive noise using a hybrid noise injection mechanism of the generalized linear model. This mechanism intelligently adjusts the noise type and intensity by analyzing data distribution characteristics, significantly improving data usability while maintaining the same level of privacy protection. The hybrid noise is a weighted combination of Laplace noise and Gaussian noise, with a weighted combination ratio of 70% Laplace noise and 30% Gaussian noise. Laplace noise satisfies strict privacy budget differential privacy protection, while Gaussian noise provides better data usability under the same privacy budget. Through weighted combination, a dynamic balance can be achieved between privacy protection strength and data utility. Since different shards have different sensitivities, the generated noise amplitude will also be adaptively adjusted to ensure compliance with a unified privacy budget constraint. Specifically, the Laplace noise addition mechanism during noise generation is represented by the following formula:

[0054] in, This indicates partitioning the original model. Model slices after adding Laplacian noise; Indicates Laplace noise, Indicates privacy budget, This indicates the sensitivity of the current model segment. The standard deviation of Gaussian noise can be calculated using the following formula: ; in, This represents the standard deviation of Gaussian noise in the i-th model segment; This represents the relaxation term, which makes the noise satisfy the differential privacy constraint; Indicates privacy budget; This represents the sensitivity of the i-th model segment.

[0055] The source node adds generated adaptive noise to the parameters of the model shards to be transmitted, achieving privacy perturbation of the original data. After adding noise, the shards are subjected to value range constraints, pruning the parameter values ​​after adding noise to a reasonable range, such as the normal value range of model parameters, to avoid affecting model usability due to parameter values ​​exceeding the valid range caused by noise addition. The final result is a noisy model shard. In medical and financial industry scenarios, the value range constraint can be [0,1]. In addition to noise addition, security technologies such as zero-knowledge verification and trusted execution environment collaborative verification processes can be applied during model shard transmission to ensure the security of model shard transmission.

[0056] This invention provides a federated model fragmentation inference method based on dynamic bandwidth awareness. In this embodiment, in response to an inference task request, task features, device status data, and network status data within a preset historical time period are acquired. The network status data is then processed using a lightweight bandwidth prediction model to obtain a bandwidth prediction value. Using this bandwidth prediction value as a network status constraint, the model fragment size is optimized to obtain the expected model fragment size. A multi-dimensional state vector is constructed based on the task features, network status data, the bandwidth prediction value, and the device status data. Based on the expected model fragment size and the multi-dimensional state vector, a fragmentation decision model determines a model fragmentation acquisition strategy from preset model fragmentation strategies with different fragmentation granularities. According to the model fragmentation acquisition strategy, at least one source node and corresponding model fragments to be transmitted for each source node are identified from the model fragmentation interaction network. A transmission request for the corresponding model fragment to be transmitted is sent to the source node, and an inference task is executed based on the model fragments received from each source node. Compared to existing technologies that employ fixed strategies, this invention dynamically adjusts the fragment acquisition strategy based on real-time changes in network status, device status, and task characteristics. This effectively solves the problems of fragment mismatch and low transmission efficiency caused by fixed strategies failing to adapt to dynamic environmental changes. It significantly reduces invalid transmission and waiting time during inference task execution, while ensuring the timeliness of model fragment transmission and the continuity of task execution.

[0057] Furthermore, as a response to the above Figure 1 The implementation of the method shown in this invention provides a federated model fragmented inference device based on dynamic bandwidth awareness, such as... Figure 3 As shown, the device includes: The bandwidth prediction module 31 is used to respond to the inference task request, acquire task characteristics, device status data, and network status data within a preset historical period, and perform prediction processing on the network status data based on a lightweight bandwidth prediction model to obtain a bandwidth prediction value. The construction module 32 is used to optimize the model fragment size with the bandwidth prediction value as a network state constraint, obtain the expected model fragment size, and construct a multi-dimensional state vector based on the task characteristics, network state data, the bandwidth prediction value and the device state data. Decision module 33 is used to determine the model sharding acquisition strategy from preset model sharding strategies with different sharding granularities based on the expected model sharding size and the multidimensional state vector, according to the sharding decision model. The inference module 34 is used to determine at least one source node and the model fragments to be transmitted corresponding to each source node from the model fragment interaction network according to the model fragment acquisition strategy, send the transmission request of the corresponding model fragment to be transmitted to the source node, and perform inference tasks based on the model fragments received from each source node.

[0058] Furthermore, the device also includes: The first receiving module is used to receive the initial lightweight bandwidth prediction model sent by the regional data center, wherein the initial lightweight bandwidth prediction model is obtained by pruning a pre-trained bidirectional long short memory neural network model. The parameter optimization module is used to optimize the model parameters of the initial lightweight bandwidth prediction model based on sliding window incremental learning according to real-time network status data, so as to obtain the change of model parameters after single-node optimization. The upload module is used to upload the model parameter changes to the regional data center, so that the regional data center can perform global model aggregation based on the model parameter changes to obtain a lightweight bandwidth prediction model for distribution to each node. The second receiving module is used to receive the lightweight bandwidth prediction model sent by the regional data center.

[0059] Furthermore, the construction module 32 includes: The sharding unit is used to perform virtual sharding on the inference model corresponding to the inference task, obtain multiple virtual model shards, and determine the expected transmission bandwidth and importance parameters corresponding to each virtual model shard. An optimization unit is used to substitute the expected transmission bandwidth and importance parameters into the bandwidth-delay joint optimization function to obtain the objective function. The objective function is then optimized with the expected fragment size as the optimization variable, aiming to minimize the result of the bandwidth-delay joint optimization function, to obtain the expected model fragment size. The bandwidth-delay joint optimization function comprises the sum of two function terms: the first term is the sum of the ratios of the importance parameters of all virtual model fragments to the expected transmission bandwidth, and the second term is the desired transmission delay. The expected transmission bandwidth is constrained by the bandwidth prediction value.

[0060] Furthermore, the decision module 33 includes: An initialization unit is used to initialize the initial selection probability of each preset model partitioning strategy in the partitioning decision model with the expected model partitioning size as a prior value, and obtain a prior probability distribution. The preset model partitioning strategy includes at least one of the following: full model transmission strategy, hierarchical partitioning strategy, channel partitioning strategy, neuron partitioning strategy, and partitioning strategy based on importance parameter proportion. The evaluation value calculation unit is used to calculate the evaluation value of each preset model sharding strategy based on the multidimensional state vector and the prior probability distribution and the sharding decision model. The multidimensional state vector includes at least one of the following: bandwidth prediction value, real-time bandwidth, network latency, packet loss rate, terminal remaining battery power, terminal memory usage rate, terminal processor load, task urgency, number of consecutive uses of the model sharding strategy, historical average transmission time, and historical transmission success rate. The determining unit is used to determine the model sharding acquisition strategy from the preset model sharding strategy based on the evaluation value.

[0061] Furthermore, the device also includes: An initialization module is used to initialize a state policy value table, wherein the state policy value table includes the state dimension corresponding to the multi-dimensional state vector and the policy dimension corresponding to the preset model slicing policy; The execution module is used to select and execute the target sharding strategy from the preset model sharding strategy based on the real-time multidimensional state vector and a greedy algorithm. The acquisition module is used to acquire the actual transmission time, actual energy consumption, and model accuracy impact factor after the target fragmentation strategy is executed; The reward value calculation module is used to substitute the actual transmission time, the actual energy consumption, and the model accuracy influence factor into the sharding strategy reward function to calculate the reward value, wherein the sharding strategy reward function includes a transmission efficiency term and an energy consumption term; The update module is used to update the strategy utility values ​​corresponding to different state strategy combinations in the state strategy value table based on the real-time multidimensional state vector, the target sharding strategy, the reward value, and the multidimensional state vector after the target sharding strategy is executed. The loop module is used to repeatedly perform the operations of selecting the target sharding strategy, calculating the reward value, and updating the state strategy value table until the strategy utility value converges.

[0062] Furthermore, the inference module 34 includes: The partitioning unit is used to logically divide the inference model corresponding to the inference task into at least one model fragment to be transmitted according to the fragmentation granularity of the model fragmentation acquisition strategy, and to configure priority labels for each model fragment to be transmitted according to the transmission priority determined by the model fragmentation acquisition strategy. The query unit is used to obtain the list of currently available source nodes through the node discovery protocol in the model sharding interaction network, and to query the model sharding cache information of each node. The matching unit is used to match at least one node that caches the corresponding model fragment to be transmitted as a source node based on the model fragment cache information, and associate the model fragment to be transmitted with the source node.

[0063] Furthermore, the inference module 34 also includes: A sensitivity calculation unit is used to calculate the sensitivity of the fragment to be transmitted based on the statistical distribution of the fragmentation parameters; The noise addition unit is used to add noise to the fragments of the model to be transmitted using a mixed noise mechanism of Laplace noise and Gaussian noise, based on the sensitivity and privacy budget. The constraint unit is used to constrain the value range of the model fragment to be transmitted after adding noise, to obtain the noisy model fragment, and to transmit the noisy model fragment to the node that issued the transmission request for the model fragment to be transmitted.

[0064] This invention provides a federated model sharding inference device based on dynamic bandwidth awareness. In embodiments of this invention, in response to an inference task request, task features, device status data, and network status data within a preset historical time period are acquired. The network status data is then processed using a lightweight bandwidth prediction model to obtain a bandwidth prediction value. Using this bandwidth prediction value as a network status constraint, the model shard size is optimized to obtain the expected model shard size. A multi-dimensional state vector is constructed based on the task features, network status data, the bandwidth prediction value, and the device status data. Based on the expected model shard size and the multi-dimensional state vector, a sharding decision model determines a model sharding acquisition strategy from preset model sharding strategies with different sharding granularities. Based on the model sharding acquisition strategy, at least one source node and corresponding model shards to be transmitted for each source node are identified from the model sharding interaction network. A transmission request for the corresponding model shard to be transmitted is sent to the source node, and an inference task is executed based on the model shards received from each source node. Compared to existing technologies that employ fixed strategies, this invention dynamically adjusts the fragment acquisition strategy based on real-time changes in network status, device status, and task characteristics. This effectively solves the problems of fragment mismatch and low transmission efficiency caused by fixed strategies failing to adapt to dynamic environmental changes. It significantly reduces invalid transmission and waiting time during inference task execution, while ensuring the timeliness of model fragment transmission and the continuity of task execution.

[0065] According to one embodiment of the present invention, a storage medium is provided, the storage medium storing at least one executable instruction that can execute the dynamic bandwidth-aware federated model fragmented inference method in any of the above method embodiments.

[0066] Figure 4 The diagram shows a structural schematic of a terminal according to an embodiment of the present invention. The specific implementation of the present invention is not limited to the specific implementation of the terminal.

[0067] like Figure 4 As shown, the terminal may include: a processor 402, a communication interface 404, a memory 406, and a communication bus 408.

[0068] The processor 402, communication interface 404, and memory 406 communicate with each other via communication bus 408.

[0069] Communication interface 404 is used for network communication with other devices such as clients or other servers.

[0070] The processor 402 is used to execute program 410, specifically to execute the relevant steps in the above embodiment of the federated model fragmented inference method based on dynamic bandwidth awareness.

[0071] Specifically, program 410 may include program code that includes computer operation instructions.

[0072] Processor 402 may be a central processing unit (CPU), a specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention. The terminal may include one or more processors of the same type, such as one or more CPUs; or it may include processors of different types, such as one or more CPUs and one or more ASICs.

[0073] Memory 406 is used to store program 410. Memory 406 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0074] Specifically, program 410 can be used to cause processor 402 to perform the following operations: In response to an inference task request, the system acquires task characteristics, device status data, and network status data within a preset historical time period. Based on a lightweight bandwidth prediction model, the system performs prediction processing on the network status data to obtain a bandwidth prediction value. Using the bandwidth prediction value as a network state constraint, the model fragment size is optimized to obtain the expected model fragment size, and a multi-dimensional state vector is constructed based on the task characteristics, network state data, the bandwidth prediction value, and the device state data. Based on the expected model shard size and the multidimensional state vector, a model shard acquisition strategy is determined from preset model sharding strategies with different sharding granularities using a sharding decision model. Based on the model fragment acquisition strategy, at least one source node and the corresponding model fragments to be transmitted for each source node are determined from the model fragment interaction network. A transmission request for the corresponding model fragment to be transmitted is sent to the source node, and an inference task is performed based on the model fragments received from each source node.

[0075] It is obvious to those skilled in the art that the modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those presented herein, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.

[0076] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A federated model slicing inference method based on dynamic bandwidth awareness, characterized in that, include: In response to an inference task request, the system acquires task characteristics, device status data, and network status data within a preset historical time period. Based on a lightweight bandwidth prediction model, the system performs prediction processing on the network status data to obtain a bandwidth prediction value. Using the bandwidth prediction value as a network state constraint, the model fragment size is optimized to obtain the expected model fragment size, and a multi-dimensional state vector is constructed based on the task characteristics, the network state data, the bandwidth prediction value, and the device state data. Based on the expected model shard size and the multidimensional state vector, a model shard acquisition strategy is determined from preset model sharding strategies with different sharding granularities using a sharding decision model. Based on the model fragment acquisition strategy, at least one source node and the model fragments to be transmitted corresponding to each source node are determined from the model fragment interaction network. A transmission request for the corresponding model fragment to be transmitted is sent to the source node, and an inference task is performed based on the model fragments received from each source node. Using the bandwidth prediction value as a network state constraint, the model fragment size is optimized to obtain the expected model fragment size, specifically including: The inference model corresponding to the inference task is virtually fragmented to obtain multiple virtual model fragments, and the expected transmission bandwidth and importance parameters corresponding to each virtual model fragment are determined. Substituting the expected transmission bandwidth and importance parameters into the bandwidth-delay joint optimization function, we obtain the objective function. With minimizing the result of the bandwidth-delay joint optimization function as the objective and the expected fragmentation size as the optimization variable, we optimize the objective function to obtain the expected model fragmentation size. The bandwidth-delay joint optimization function includes the sum of two function terms: the first function term is the sum of the ratios of the importance parameters of all virtual model fragments to the expected transmission bandwidth, and the second function term is the expected transmission delay, wherein the expected transmission bandwidth is constrained by the bandwidth prediction value. Based on the expected model shard size and the multidimensional state vector, a model shard acquisition strategy is determined from preset model sharding strategies with different sharding granularities using a sharding decision model. Specifically, this includes: Using the expected model partition size as a prior value, the initial selection probability of each preset model partitioning strategy in the partitioning decision model is initialized to obtain the prior probability distribution. The preset model partitioning strategy includes at least one of the following: full model transmission strategy, hierarchical partitioning strategy, channel partitioning strategy, neuron partitioning strategy, and partitioning strategy based on importance parameter proportion. Based on the multidimensional state vector and the prior probability distribution, the evaluation value of each preset model sharding strategy is calculated based on the sharding decision model. The multidimensional state vector includes at least one of the following: bandwidth prediction value, real-time bandwidth, network latency, packet loss rate, terminal remaining battery power, terminal memory usage rate, terminal processor load, task urgency, number of consecutive uses of the model sharding strategy, historical average transmission time, and historical transmission success rate. The model sharding acquisition strategy is determined from the preset model sharding strategy based on the evaluation value.

2. The method for fragmented inference of a federated model based on dynamic bandwidth awareness according to claim 1, characterized in that, Before performing the bandwidth prediction process on the network state data based on the lightweight bandwidth prediction model to obtain the bandwidth prediction value, the method further includes: The initial lightweight bandwidth prediction model is received from the regional data center, wherein the initial lightweight bandwidth prediction model is obtained by pruning a pre-trained bidirectional long short memory neural network model. Based on real-time network status data, the model parameters of the initial lightweight bandwidth prediction model are optimized using sliding window incremental learning to obtain the change in model parameters after single-node optimization. The model parameter changes are uploaded to the regional data center so that the regional data center can perform global model aggregation based on the model parameter changes to obtain a lightweight bandwidth prediction model for distribution to each node. Receive the lightweight bandwidth prediction model issued by the regional data center.

3. The method for fragmented inference of a federated model based on dynamic bandwidth awareness according to claim 1, characterized in that, Based on the expected model shard size and the multidimensional state vector, before determining the model shard acquisition strategy from preset model sharding strategies with different sharding granularities using the sharding decision model, the method further includes: Initialize the state policy value table, wherein the state policy value table includes the state dimension corresponding to the multi-dimensional state vector and the policy dimension corresponding to the preset model slicing policy; Based on the real-time multidimensional state vector, a target sharding strategy is selected and executed from the preset model sharding strategy using a greedy algorithm; Obtain the actual transmission time, actual energy consumption, and model accuracy impact factor after the target fragmentation strategy is executed; The actual transmission time, the actual energy consumption, and the model accuracy impact factor are substituted into the sharding strategy reward function to calculate the reward value, wherein the sharding strategy reward function includes a transmission efficiency term and an energy consumption term; Based on the real-time multidimensional state vector, the target sharding strategy, the reward value, and the multidimensional state vector after the target sharding strategy is executed, update the strategy utility values ​​corresponding to different state strategy combinations in the state strategy value table. Repeat the operations of selecting the target sharding strategy, calculating the reward value, and updating the state strategy value table until the strategy utility value converges.

4. The method for fragmented inference of a federated model based on dynamic bandwidth awareness according to claim 1, characterized in that, Based on the model fragment acquisition strategy, at least one source node and the corresponding model fragments to be transmitted for each source node are determined from the model fragment interaction network, including: According to the granularity of the model fragment acquisition strategy, the inference model corresponding to the inference task is logically divided into at least one model fragment to be transmitted, and priority labels are configured for each model fragment to be transmitted according to the transmission priority determined by the model fragment acquisition strategy. In the model sharding interaction network, the list of currently available source nodes is obtained through the node discovery protocol, and the model sharding cache information of each node is queried. Based on the model sharding cache information, at least one node that caches the corresponding model shard to be transmitted is matched as the source node for the model shard to be transmitted, and the model shard to be transmitted is associated with the source node.

5. The method for fragmented inference of a federated model based on dynamic bandwidth awareness according to claim 1, characterized in that, For any of the aforementioned source nodes, the transmission process of the model fragment to be transmitted includes: The sensitivity of the fragment to be transmitted is calculated based on the statistical distribution of the fragmentation parameters; Based on the aforementioned sensitivity and privacy budget, a mixed noise mechanism of Laplace noise and Gaussian noise is used to add noise to the fragments of the model to be transmitted; The noise-added model fragments to be transmitted are subjected to range constraints to obtain noisy model fragments, and the noisy model fragments are transmitted to the node that issued the transmission request for the model fragments to be transmitted.

6. A fragmented inference device for a federated model based on dynamic bandwidth awareness, characterized in that, The apparatus is used to perform the operations corresponding to the dynamic bandwidth-aware federated model fragmented inference method as described in any one of claims 1-5, including: The bandwidth prediction module is used to respond to inference task requests, acquire task characteristics, device status data, and network status data within a preset historical time period, and perform prediction processing on the network status data based on a lightweight bandwidth prediction model to obtain bandwidth prediction values. The module is used to optimize the model fragment size with the bandwidth prediction value as a network state constraint, obtain the expected model fragment size, and construct a multi-dimensional state vector based on the task characteristics, the network state data, the bandwidth prediction value, and the device state data. The decision module is used to determine the model sharding acquisition strategy from preset model sharding strategies with different sharding granularities based on the expected model sharding size and the multidimensional state vector, according to the sharding decision model. The inference module is used to determine at least one source node and the corresponding model fragments to be transmitted from the model fragment interaction network according to the model fragment acquisition strategy, send the corresponding model fragment to be transmitted to the source node, and perform inference tasks based on the model fragments received from each source node.

7. A storage medium, characterized in that, The storage medium stores at least one executable instruction that causes the processor to perform the operation corresponding to the dynamic bandwidth-aware federated model fragmented inference method as described in any one of claims 1-5.

8. A terminal, characterized in that, include: The processor, memory, communication interface, and communication bus are provided, wherein the processor, memory, and communication interface communicate with each other via the communication bus. The memory is used to store at least one executable instruction that causes the processor to perform the operation corresponding to the dynamic bandwidth-aware federated model fragmented inference method as described in any one of claims 1-5.

Images