Federal learning-based joint optimization method for common sensing algorithm fusion system resources
By constructing a federated learning-based sensory-computation fusion system, the system optimizes perception quality, communication resources, and computing capabilities, solving the problems of perception data pollution and strong resource coupling. This enables stable convergence and low latency of federated learning in complex environments, improving the operational efficiency and reliability of intelligent systems.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- NORTHEASTERN UNIV CHINA
- Filing Date
- 2026-02-05
- Publication Date
- 2026-05-19
AI Technical Summary
Existing technologies lack a unified optimization method that simultaneously considers the quality of perceived data, stable convergence of federated learning, and end-to-end system latency in complex and dynamic environments, making it difficult to provide reliable support for intelligent systems.
A sensor-computing fusion system based on federated learning is constructed. By building an integrated signal and communication model, a sensing model, and a computing and energy consumption model, a joint optimization problem is established and solved using an alternating iterative optimization method, including optimal device selection, computing resource allocation, and beamforming optimization.
It improves the convergence efficiency and accuracy of federated learning models, reduces end-to-end system latency, enhances system stability in various resource-constrained scenarios, and provides reliable support for edge intelligence applications.
Smart Images
Figure CN122064490A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of synesthetic computing fusion technology, and relates to a joint resource optimization method for synesthetic computing fusion systems based on federated learning. Background Technology
[0002] With the rapid development of deep learning and artificial intelligence technologies, applications such as smart healthcare, smart homes, and the Industrial Internet of Things (IIoT) are placing higher demands on efficient and low-latency intelligent services. In these applications, numerous terminal devices continuously generate high-dimensional, multimodal sensory data. This data is not only massive in scale but also typically has strong privacy sensitivity and timeliness requirements. Traditional centralized learning methods require uploading raw data to remote cloud servers for unified processing, which not only easily leads to data leakage risks but is also limited by network bandwidth and transmission latency, making it difficult to meet the application requirements of real-time decision-making and rapid response. Therefore, how to achieve low-latency, highly reliable intelligent computing services while ensuring data privacy and security has become a critical technical problem that urgently needs to be solved in the field of network and intelligent systems.
[0003] Federated Edge Learning (FEEL), a novel distributed learning paradigm, allows terminals or edge nodes to train models locally using private data, uploading only model parameters or gradient information to the central node. This avoids the direct transmission of raw data and effectively alleviates the conflict between data privacy protection and communication load. The Sensor-Computation Fusion (ISCC) framework integrates sensing, communication, and computing capabilities, constructing a closed-loop system of "sensing-transmission-computation," providing system-level support for FEEL deployment in edge environments. The ISCC framework can simultaneously complete environmental perception, model training, and parameter transmission within the same system, providing unified infrastructure support for federated learning for intelligent applications.
[0004] However, while the deep integration of ISCC and FEEL improves system efficiency, it also introduces a series of new technical challenges. In ISCC systems sharing wireless resources, data pollution caused by sensing interference, the high sensitivity of federated learning to data quality, and the strong coupling between sensing and computing resources constitute the core bottlenecks restricting the stable and efficient operation of FEEL. Current technologies lack a unified optimization method that can simultaneously address sensing data quality, stable convergence of federated learning, and end-to-end system latency in complex dynamic environments, making it difficult to provide reliable support for intelligent systems geared towards practical applications. Summary of the Invention
[0005] To address the aforementioned technical problems, the purpose of this invention is to provide a joint resource optimization method for a synesthetic computing fusion system based on federated learning.
[0006] This invention provides a method for joint resource optimization of a synesthetic computing fusion system based on federated learning, comprising:
[0007] Step 1: Construct a synesthetic computing fusion system based on federated learning;
[0008] Step 2: Construct an integrated signal and communication model, sensing model, and computing and energy consumption model;
[0009] Step 3: Construct a joint optimization problem while satisfying constraints on perception quality, communication, computation, and energy;
[0010] Step 4: Decompose the joint optimization problem into three sub-problems: equipment selection optimization, computational resource allocation optimization, and joint beamforming optimization. Use an alternating iterative optimization method to solve the three types of sub-problems and output the optimal equipment selection variable, the optimal CPU computing frequency of the equipment, and the optimal communication beamforming vector and sensing beamforming matrix.
[0011] The present invention provides a joint resource optimization method for a synesthetic computing fusion system based on federated learning, which has the following beneficial effects:
[0012] This invention improves the convergence efficiency and accuracy of federated learning models by jointly optimizing sensing quality, communication resources and computing power in a fusion network of sensing, communication and computing. It also effectively reduces end-to-end latency and enhances system stability in various resource-constrained scenarios, providing reliable technical support for edge intelligence applications in complex and dynamic environments. Attached Figure Description
[0013] Figure 1 This is a flowchart of a joint resource optimization method for a synesthetic computing fusion system based on federated learning, according to the present invention.
[0014] Figure 2 The number of different ISCC devices in the embodiments of the present invention The following is a comparison chart of test accuracy;
[0015] Figure 3 These are different trade-off coefficients in the embodiments of the present invention. Comparison of test accuracy;
[0016] Figure 4 This is a comparison of system delay with the RARA scheme under different maximum transmit power constraints in the embodiments of the present invention;
[0017] Figure 5 This is a comparison of system latency with the RARA scheme under different maximum computation frequency constraints in the embodiments of the present invention;
[0018] Figure 6 This is a comparison of the mutual information probability distribution between the method of this invention and the RARA scheme;
[0019] Figure 7 This is a comparison of system latency under different upload strategies in the embodiments of the present invention. Detailed Implementation
[0020] To address the issues of sensory data contamination and strong resource coupling, this invention proposes a joint resource optimization method for a sensory-computer fusion system based on federated learning. The core of this invention lies in using sensory mutual information as a threshold for data quality, establishing a quantitative relationship between data quality, device selection, multi-dimensional resource allocation, and the convergence performance of federated learning, thereby constructing a unified joint optimization problem, and employing a hierarchical alternating optimization framework for efficient solution. Ultimately, this achieves stable convergence of federated learning under complex interference environments and low system latency. Specifically, the method is implemented through the following four steps:
[0021] like Figure 1 As shown, the present invention provides a method for joint resource optimization of a synesthetic computing fusion system based on federated learning, comprising:
[0022] Step 1: Construct a synesthetic computing fusion system based on federated learning.
[0023] The federated learning-based synergistic computing system includes: a base station equipped with a single antenna and connected to an edge server, and Each equipment The ISCC device of the meta-antenna array; the base station deploys a global model, and each ISCC device deploys a local model; each ISCC device uses the same hardware platform and wireless resources to synchronously perform radar environmental perception of the predetermined target, local model training based on perception data, and uploading of model parameters.
[0024] The federated learning model training consists of multiple communication rounds. In the federated learning model initialization phase, the base station broadcasts the global model to all ISCC devices; each ISCC device uses all power resources to perform radar environmental perception and collect data samples; then each device uses computing resources to perform local model updates.
[0025] After the federated learning model is initialized, each round of communication mainly includes the following three steps:
[0026] (1) ISCC devices simultaneously perform radar environment perception and local model upload: For model upload, each ISCC device will upload the previous round of local model. Transmitted to the base station; for radar environmental perception, the echo signal from each target is converted into a set of radar data, used... This indicates the number of echo signal samples collected by ISCC device k.
[0027] (2) After receiving the updated local models from all participating ISCC devices, the global model is aggregated and updated, and then broadcast to each ISCC device; the global model aggregation update is as follows:
[0028]
[0029]
[0030] in, This is an abstract indicator variable for whether a device is selected to participate in scheduling, i.e., if and only if the perceived mutual information of device k is... Exceeding the preset threshold hour, , indicating equipment Selected to participate in scheduling.
[0031] (3) Each ISCC device updates its local model using the stochastic gradient descent method based on the echo signal samples acquired in the current round and the received global model.
[0032] Step 2: Construct an integrated signal and communication model, a sensing model, and a computation and energy consumption model, specifically as follows:
[0033] Step 2.1: Construct an integrated signal and communication model:
[0034] To achieve the integration of sensing and communication functions, the ISCC device k transmits signals. Designed as a weighted combination of communication flow and sensing flow:
[0035]
[0036] in, Assign a beamforming vector to the communication beam of device k. Let k be the transmission signal vector of ISCC device k. For device k, the sensing beamforming matrix, For the ISCC device k radar sensing waveform vector, satisfy .
[0037] The signal received by the base station is:
[0038]
[0039] in, This represents the uplink channel vector from ISCC device k to the base station. It is additive white Gaussian noise. The communication variance is the uplink communication rate of device k. for:
[0040]
[0041] in, For system bandwidth; The symbol represents the uplink channel vector from ISCC device i to the base station; Assign a beamforming vector to device i; Sensing beamforming matrix for device i; This represents the communication noise power.
[0042] ISCC equipment Transmission power The trace of the covariance of the transmitted waveform:
[0043]
[0044] in, This represents the covariance of the transmitted waveform.
[0045] Step 2.2: Constructing the perception model:
[0046] The target echo signal received by ISCC device k is:
[0047]
[0048] in, The complex reflection coefficient of the target. For array guide vector, Let be the azimuth angle of the target relative to the antenna array of ISCC device k. Let represent the sensing interference channel matrix of device i to device k. To sense noise.
[0049] Using perceived mutual information as the core indicator, the information quality of perceived data under interference and noise is quantified:
[0050]
[0051] in, , The transmitted signal covariance matrix, For the sensing mutual information of device k. The interference plus noise covariance matrix; ; The perceived noise power.
[0052] Step 2.3: Constructing the computation and energy consumption model:
[0053] The computational latency for device k to complete local model training is:
[0054]
[0055] in, This represents the number of local echo signal samples. The number of CPU cycles required to process a single sample. Calculate the CPU frequency for device k; device Upload The communication delay for the bit model parameters is:
[0056]
[0057] Considering synchronous federated learning, the total time of a single round of global iteration is determined by the slowest device, and the system end-to-end latency is:
[0058]
[0059] in, For the set of devices selected to participate in the aggregation in this round, the total energy consumption of device k in a single iteration is:
[0060]
[0061] in, The effective capacitance coefficient is determined by the chip architecture.
[0062] Step 3: Under the premise of satisfying the constraints of perception quality, communication, computation, and energy, construct a joint optimization problem, specifically as follows:
[0063] We construct a joint optimization problem that aims to balance the convergence performance of federated learning with the end-to-end latency of the system, while satisfying constraints on perception quality, communication, computation, and energy. This problem is formalized as follows:
[0064]
[0065]
[0066]
[0067]
[0068]
[0069]
[0070]
[0071] in, For the weighting factor; This indicates the number of echo signal samples collected by ISCC device k. A set of communication beamforming vectors for all devices. A set of sensing beamforming matrices for all devices. Calculate the set of CPU frequencies for all devices. For the set of binary decision variables for all devices, Let k be the maximum total energy consumption of device k in a single iteration. The maximum CPU computing frequency of device k; For equipment The maximum transmit power; M is the number of devices to be scheduled; Let be a binary decision variable for device k, representing whether device k is selected to participate in scheduling. The time-representation device k is selected to participate in the scheduling, with the following constraints: This ensured the quality of the sensing data from the participating devices; when When device k is not selected to participate in scheduling, the constraint degenerates into... This condition always holds true, which is equivalent to removing the perception quality limitation for unselected devices.
[0072] Step 4: Decompose the joint optimization problem into three sub-problems: equipment selection optimization, computational resource allocation optimization, and joint beamforming optimization. Solve these three sub-problems using an alternating iterative optimization method, outputting the optimal equipment selection variable, the optimal CPU computing frequency for each equipment, and the optimal communication beamforming vector and sensing beamforming matrix. Specifically:
[0073] Step 4.1: Decompose the joint optimization problem P1 of mixed integer nonlinear programming into equipment selection optimization subproblem, computational resource allocation optimization subproblem and joint beamforming optimization subproblem, and initialize equipment selection variables, equipment CPU computing frequency, communication beamforming vector, sensing beamforming matrix and maximum number of iterations.
[0074] Step 4.2: Fix the communication beamforming vector, the sensing beamforming matrix, and the CPU calculation frequency. Introduce a binary selection variable to indicate whether the terminal device participates in federated learning. Transform the integer programming problem into a continuous optimization problem through a quadratic constraint equivalent transformation. Based on convex relaxation and the augmented Lagrange multiplier method, solve the device selection optimization subproblem to obtain the optimal device selection variable. Specifically:
[0075] Given a fixed communication beamforming vector, a sensing beamforming matrix, and a CPU computing frequency, the equipment selection optimization subproblem is formulated as follows:
[0076]
[0077]
[0078]
[0079]
[0080] Subproblems For an integer programming problem, according to Lemma 1, define the following set:
[0081] like Then there must be , yes The set, ,and .
[0082] At this point, the original integer programming problem It can be equivalently transformed into the following continuous optimization form:
[0083]
[0084]
[0085]
[0086]
[0087]
[0088]
[0089] Regarding the issues after conversion The equality constraints in the equation are solved using the augmented Lagrange multiplier method, and the equality constraint function is defined as follows:
[0090]
[0091] The augmented Lagrange function is then:
[0092]
[0093] in The original objective function is... For Lagrange multipliers, The penalty parameter; the augmented Lagrange multiplier method updates variables alternately. , and the vehicle To approximate the optimal solution; in the first In this iteration, the update steps are as follows:
[0094] Update b:
[0095]
[0096]
[0097] renew :
[0098]
[0099]
[0100] Update the Lagrange multipliers:
[0101]
[0102] The updated device selection variables are obtained through the above process.
[0103] Step 4.3: Fixed equipment selection variables, communication beamforming vectors, and sensing beamforming matrices are used to minimize the maximum completion latency of all participating terminal devices in a single round of federated learning. Based on the terminal device's energy budget, local computing task size, and hardware capability limits, the optimal computing frequency for each terminal device is adaptively determined. The computational resource allocation optimization sub-problem is solved to optimize the computing frequency of each participating scheduling device. Specifically:
[0104] Given fixed equipment selection variables, communication beamforming vectors, and sensing beamforming matrices, the computational resource allocation subproblem aims to minimize the maximum completion time of all participating devices in a single round of federated learning by rationally allocating computational resources among the devices, thereby reducing end-to-end latency. The computational resource allocation optimization subproblem is specifically formulated as follows:
[0105]
[0106]
[0107]
[0108] in, For the selected subset of devices to be scheduled, due to the objective function Let $k$ be the maximum completion time for each device, and $k$ be the computational delay for any device $k$ to complete local model training. Follow The maximum completion time increases and then monotonically decreases; therefore, to minimize the maximum completion time, higher CPU frequencies should be allocated to each device as much as possible; combined with energy constraints... Derive the upper bound of the CPU calculation frequency for each device, and substitute the energy expression to obtain the constraint:
[0109]
[0110] The results were:
[0111]
[0112] This upper bound is determined by the device's available remaining energy, i.e., total energy minus transmission energy consumption, and the scale of the local computing task; in addition, the frequency must also meet hardware capability constraints. Therefore, under the premise of simultaneously satisfying energy and hardware constraints, the objective function is to... The optimal CPU computing frequency should be the smaller of the two values, i.e.:
[0113]
[0114] in, The optimal CPU calculation frequency for device k.
[0115] Step 4.4: Given the device selection variables and the device's CPU computing frequency, and considering the non-convex nature of the sensing mutual information constraint and communication rate constraint, auxiliary variables are introduced, and a successive convex approximation method is used to transform the original problem into a semidefinite programming problem. Then, the joint beamforming optimization subproblem is solved, jointly optimizing the communication beamforming vector and the sensing beamforming matrix, specifically:
[0116] Given the device selection variables and the device's CPU computing frequency, the communication beamforming vector and the sensing beamforming matrix are jointly optimized. The joint beamforming optimization subproblem is expressed as follows:
[0117]
[0118]
[0119]
[0120]
[0121] Among them, the total system latency , For the selected subset of devices to be scheduled, the objective function is... The goal is to minimize the end-to-end latency determined by the slowest device; this is a non-convex optimization problem. To apply efficient convex optimization tools, the non-convex problem must be approximated. First, auxiliary variables are introduced. Using epigraph transformation to solve the original problem Equivalent transformation to:
[0122]
[0123]
[0124]
[0125]
[0126]
[0127] Despite the equivalent transformation, the constraint non-convexity remains unchanged. Therefore, successive convex approximation is adopted, which uses a series of convex approximations to approximate the optimal solution of the original problem.
[0128] Convexify the energy constraint:
[0129] Total energy consumption of equipment middle, and non-convex Related; to decouple, the arithmetic-geometric mean inequality is used to link the constraints. Transform into a topic about and transmission power Second-order cone constraint:
[0130]
[0131] Introducing positive semidefinite matrices and Uplink communication rate rewritten in differential form:
[0132]
[0133] make:
[0134]
[0135] The first term is a concave function, and the second term... It is a convex function; since the negative value of a concave function is convex, therefore It is the difference between concave and convex functions, and the whole is non-convex; using The concavity at point Performing a first-order Taylor expansion at this point, we obtain its upper bound:
[0136] (29)
[0137] Substituting (29) into (27), we obtain the rate. A convex lower bound approximation :
[0138]
[0139] Introduce the covariance matrix and use the matrix identity:
[0140] and .
[0141] Rewrite it in difference convex form:
[0142]
[0143] In this formula, the first term is a concave function and the second term is a convex function;
[0144]
[0145] Similar to handling uplink communication rates, the convex terms in equation (31) are... At the reference point Performing a first-order Taylor expansion at this point, we obtain its upper bound:
[0146]
[0147] Substituting into (31), we obtain mutual information. A convex lower bound approximation :
[0148]
[0149] in Since is a constant, the following convex optimization problem is obtained:
[0150]
[0151]
[0152]
[0153]
[0154] The question The optimal communication beamforming vector and sensing beamforming matrix can be obtained by using convex optimization tools.
[0155] Step 4.5: Iterate and optimize the equipment selection optimization subproblem, the computing resource allocation optimization subproblem, and the joint beamforming optimization subproblem alternately until each optimization subproblem converges, and output the optimal equipment selection variable, the optimal CPU computing frequency of the equipment, the communication beamforming vector, and the sensing beamforming matrix.
[0156] Example
[0157] This embodiment is based on a simulation of a FEEL system integrating sensing, communication, and computing. The system includes a single-antenna base station and multiple ISCC devices equipped with antenna arrays. The base station is located at coordinates (0,0,8) m. Several ISCC devices are randomly distributed within a circular area with a radius of 500 m centered on the base station. Each ISCC device is equipped with... One antenna, simultaneously responsible for monitoring a distance of 200 meters ahead at an azimuth angle. The system performs radar sensing of targets and uses the sensing data to train a local federated learning model. The communication channel in the system employs a Rician fading model, the reflection coefficient of the sensed targets follows a complex Gaussian distribution, and the interference channel between devices is modeled as a complex Gaussian matrix. The federated learning model uses a fully connected network with 50 neurons and is trained and tested using the MNIST dataset.
[0158] like Figure 2 The figure shows the number of different ISCC devices. In a comparison of test accuracy under different ISCC device participation numbers, the proposed algorithm for joint device selection and computational resource allocation, compared to a scheme that only selects devices without optimizing resource allocation (i.e., the Random Resource Allocation Strategy based on Perceived Quality Constraints (RdUS),) achieves stable model convergence in fewer communication rounds and obtains higher test accuracy. The improvement in model accuracy is particularly significant when the number of participating devices increases from a small to a large number, indicating that by reasonably introducing more devices that meet perceived quality requirements into the training process, this invention effectively enhances the model's learning ability to diverse data distributions and improves its generalization performance.
[0159] Among them, the Random resource allocation with sensing Quality constraint (RdUS) introduces sensing quality constraints during the device selection phase to ensure that the sensing data of devices participating in federated learning meets the minimum quality requirements. Specifically, this strategy only selects devices with sensing mutual information. Not lower than the set threshold The ISCC equipment is used in training, thereby reducing model update bias caused by perceptual interference at the data source. However, after satisfying this constraint, other system resources, including computation frequency, are affected. Transmission power Communication beamforming vector With sensing beamforming matrix All assignments were randomized, without further consideration of system latency, energy consumption, or resource coupling. The purpose of this strategy is to verify that relying solely on perceived quality screening without resource co-optimization makes it difficult to achieve efficient and stable federated learning in an ISCC environment.
[0160] like Figure 3 As shown, by adjusting the trade-off coefficient between convergence performance and system delay in the optimization objective... This invention enables flexible adjustment of the system's operating state under different application requirements. When the tradeoff factor favors model accuracy, the system prioritizes the participation of more ISCC devices in federated training, thereby significantly improving the final model accuracy. When the tradeoff factor favors low latency requirements, the system effectively reduces overall latency by limiting the number of participating nodes and optimizing resource allocation. These results demonstrate that this invention is applicable to both real-time tasks and high-precision intelligent analysis scenarios.
[0161] like Figure 4 As shown, under different maximum transmit power constraints, with the increase in the number of ISCC devices, the system faces more severe communication and sensing interference, resulting in a decrease in overall transmission rate and an increase in model upload latency. Compared to the random device selection and random technical resource allocation scheme, i.e., the completely random device selection and resource allocation strategy (RARA), this invention can significantly reduce the total system latency under various device scale conditions by dynamically coordinating communication power and computing resources. Furthermore, when the maximum transmit power constraint is relaxed, this invention can further improve resource utilization efficiency, thereby effectively shortening the overall time spent on model training and uploading.
[0162] Among them, the Fully Random Device Selection and Resource Allocation (RARA) strategy serves as a completely unoptimized benchmark, employing random mechanisms in both device selection and all resource dimensions. This strategy randomly determines the ISCC devices participating in aggregation in each training round and randomly allocates their computational, communication, and sensing resources, without imposing any sensing quality, latency, or energy constraints. RARA reflects the lower performance bound that FEEL might exhibit in an ISCC system without introducing any collaborative optimization mechanisms. It primarily serves to highlight the necessity of the proposed joint optimization framework, especially its significant advantages in ensuring sensing data quality, multi-resource collaborative scheduling, and improving overall system efficiency.
[0163] like Figure 5 As shown, under different maximum computational frequency constraints, the method proposed in this invention can effectively reduce the end-to-end latency of the system. When computing power is increased, the local model training time is significantly shortened, thereby reducing the latency of bottleneck nodes in the entire federated learning process. Compared with the scheme without joint optimization, this invention exhibits superior system latency performance under various computing power conditions, indicating that computing resources have a crucial impact on the overall performance of the synesthetic computing fusion system, and this invention can effectively schedule these resources.
[0164] like Figure 6As shown, perceptual mutual information, as an important indicator for measuring the quality of perceptual data, reaches a preset threshold with a significantly higher probability under the method of this invention compared to schemes without joint optimization. This result demonstrates that this invention effectively suppresses the impact of communication interference on perceptual data by coordinating resource allocation between perceptual and communication signals, thereby reducing the probability of training data contamination, providing more reliable data input for federated learning, and preventing model performance degradation due to perceptual errors.
[0165] like Figure 7 As shown, under different energy budgets and different numbers of ISCC devices, the present invention consistently outperforms the comparative schemes in terms of total system latency. When the energy budget increases, the performance gap between the schemes gradually narrows, indicating that the resource bottleneck is alleviated. Even under energy-constrained conditions, the present invention can still significantly reduce local training time and model upload time by jointly optimizing communication rate and computation frequency, demonstrating good robustness and stability.
[0166] The above description is only a preferred embodiment of the present invention and is not intended to limit the ideas of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A method for joint resource optimization in a synesthetic computing fusion system based on federated learning, characterized in that, include: Step 1: Construct a synesthetic computing fusion system based on federated learning; Step 2: Construct an integrated signal and communication model, sensing model, and computing and energy consumption model; Step 3: Construct a joint optimization problem while satisfying constraints on perception quality, communication, computation, and energy; Step 4: Decompose the joint optimization problem into three sub-problems: equipment selection optimization, computational resource allocation optimization, and joint beamforming optimization. Use an alternating iterative optimization method to solve the three types of sub-problems and output the optimal equipment selection variable, the optimal CPU computing frequency of the equipment, and the optimal communication beamforming vector and sensing beamforming matrix.
2. The method for joint resource optimization of a synesthetic computing fusion system based on federated learning according to claim 1, characterized in that, The federated learning-based sensory computing fusion system in step 1 includes: a base station equipped with a single antenna and connected to an edge server and Each equipment The ISCC device of the antenna array; the base station deploys a global model, and each ISCC device deploys a local model; each ISCC device uses the same hardware platform and wireless resources to synchronously perform radar environmental perception of the predetermined target, local model training based on perception data, and uploading of model parameters. The federated learning model training consists of multiple communication rounds. In the federated learning model initialization phase, the base station broadcasts the global model to all ISCC devices; each ISCC device uses all power resources to perform radar environmental perception and collect data samples; then each device uses computing resources to perform local model updates. After the federated learning model is initialized, each round of communication mainly includes the following three steps: (1) ISCC devices simultaneously perform radar environment perception and local model upload: For model upload, each ISCC device will upload the previous round of local model. Transmitted to the base station; for radar environmental perception, the echo signal from each target is converted into a set of radar data, used... This indicates the number of echo signal samples collected by ISCC device k; (2) After receiving the updated local models from all participating ISCC devices, the global model is aggregated and updated, and then broadcast to each ISCC device; the global model aggregation update is as follows: in, This is an abstract indicator variable for whether a device is selected to participate in scheduling, i.e., if and only if the perceived mutual information of device k is... Exceeding the preset threshold hour, , indicating equipment Selected to participate in scheduling; (3) Each ISCC device updates its local model using the stochastic gradient descent method based on the echo signal samples acquired in the current round and the received global model.
3. The method for joint resource optimization of a synesthetic computing fusion system based on federated learning according to claim 1, characterized in that, Step 2 specifically involves: Step 2.1: Construct an integrated signal and communication model: To achieve the integration of sensing and communication functions, the ISCC device k transmits signals. Designed as a weighted combination of communication flow and sensing flow: in, Assign a beamforming vector to the communication beam of device k. Let k be the transmission signal vector of ISCC device k. For device k, the sensing beamforming matrix, For the ISCC device k radar sensing waveform vector, satisfy ; The signal received by the base station is: in, This represents the uplink channel vector from ISCC device k to the base station. It is additive white Gaussian noise. The communication variance is the uplink communication rate of device k. for: in, For system bandwidth; The symbol represents the uplink channel vector from ISCC device i to the base station; Assign a beamforming vector to device i; Sensing beamforming matrix for device i; Communication noise power; ISCC equipment Transmission power The trace of the covariance of the transmitted waveform: in, Represents the covariance of the transmitted waveform; Step 2.2: Constructing the perception model: The target echo signal received by ISCC device k is: in, The complex reflection coefficient of the target. For array guide vector, Let be the azimuth angle of the target relative to the antenna array of ISCC device k. Let represent the sensing interference channel matrix of device i to device k. To sense noise; Using perceived mutual information as the core indicator, the information quality of perceived data under interference and noise is quantified: in, , Let the transmitted signal covariance matrix be... For the sensing mutual information of device k. The interference plus noise covariance matrix; ; The perceived noise power; Step 2.3: Constructing the computation and energy consumption model: The computational latency for device k to complete local model training is: in, This represents the number of local echo signal samples. The number of CPU cycles required to process a single sample. Calculate the CPU frequency for device k; device Upload The communication delay for the bit model parameters is: Considering synchronous federated learning, the total time of a single round of global iteration is determined by the slowest device, and the system end-to-end latency is: in, For the set of devices selected to participate in the aggregation in this round, the total energy consumption of device k in a single iteration is: in, The effective capacitance coefficient is determined by the chip architecture.
4. The method for joint resource optimization of a synesthetic computing fusion system based on federated learning according to claim 3, characterized in that, Step 3 specifically involves: We construct a joint optimization problem that aims to balance the convergence performance of federated learning with the end-to-end latency of the system, while satisfying constraints on perception quality, communication, computation, and energy. This problem is formalized as follows: in, For the weighting factor; This indicates the number of echo signal samples collected by ISCC device k. A set of communication beamforming vectors for all devices. A set of sensing beamforming matrices for all devices. Calculate the set of CPU frequencies for all devices. For the set of binary decision variables for all devices, Let k be the maximum total energy consumption of device k in a single iteration. The maximum CPU computing frequency of device k; For equipment The maximum transmit power; M is the number of devices to be scheduled; Let be a binary decision variable for device k, representing whether device k is selected to participate in scheduling. The time-representation device k is selected to participate in the scheduling, with the following constraints: This ensured the quality of the sensing data from the participating devices; when When device k is not selected to participate in scheduling, the constraint degenerates into... This condition always holds true, which is equivalent to removing the perception quality limitation for unselected devices.
5. The method for joint resource optimization of a synesthetic computing fusion system based on federated learning according to claim 4, characterized in that, Step 4 specifically involves: Step 4.1: Decompose the joint optimization problem P1 of mixed integer nonlinear programming into equipment selection optimization subproblem, computational resource allocation optimization subproblem and joint beamforming optimization subproblem, and initialize equipment selection variables, equipment CPU computing frequency, communication beamforming vector, sensing beamforming matrix and maximum number of iterations; Step 4.2: Fix the communication beamforming vector, the sensing beamforming matrix, and the CPU calculation frequency. Based on the convex relaxation and augmented Lagrange multiplier method, solve the equipment selection optimization subproblem to obtain the optimal equipment selection variables. Step 4.3: Select variables for fixed equipment, communication beamforming vector, and sensing beamforming matrix, solve the computational resource allocation optimization subproblem, and optimize the computational frequency of each participating scheduling equipment; Step 4.4: Given the device selection variables and the device's CPU computing frequency, solve the joint beamforming optimization subproblem using the continuous convex approximation method, and jointly optimize the communication beamforming vector and the sensing beamforming matrix; Step 4.5: Iterate and optimize the equipment selection optimization subproblem, the computing resource allocation optimization subproblem, and the joint beamforming optimization subproblem alternately until each optimization subproblem converges, and output the optimal equipment selection variable, the optimal CPU computing frequency of the equipment, the communication beamforming vector, and the sensing beamforming matrix.
6. The method for joint resource optimization of a synesthetic computing fusion system based on federated learning according to claim 5, characterized in that, Step 4.2 specifically involves: Given a fixed communication beamforming vector, a sensing beamforming matrix, and a CPU computing frequency, the equipment selection optimization subproblem is formulated as follows: Subproblems For an integer programming problem, according to Lemma 1, define the following set: ， like Then there must be , yes The set, ,and ; At this point, the original integer programming problem It can be equivalently transformed into the following continuous optimization form: Regarding the issues after conversion The equality constraints in the equation are solved using the augmented Lagrange multiplier method, and the equality constraint function is defined as follows: The augmented Lagrange function is then: in The original objective function is... For Lagrange multipliers, The penalty parameter; the augmented Lagrange multiplier method updates variables alternately. , and the vehicle To approximate the optimal solution; in the first In this iteration, the update steps are as follows: Update b: renew : Update the Lagrange multipliers: The updated device selection variables are obtained through the above process.
7. The method for joint resource optimization of a synesthetic computing fusion system based on federated learning according to claim 5, characterized in that, Step 4.3 specifically involves: Given fixed equipment selection variables, communication beamforming vectors, and sensing beamforming matrices, the computational resource allocation subproblem aims to minimize the maximum completion time of all participating devices in a single round of federated learning by rationally allocating computational resources among the devices, thereby reducing end-to-end latency. The computational resource allocation optimization subproblem is specifically formulated as follows: in, For the selected subset of devices to be scheduled, due to the objective function Let $k$ be the maximum completion time for each device, and $k$ be the computational delay for any device $k$ to complete local model training. Follow The maximum completion time increases and then monotonically decreases; therefore, to minimize the maximum completion time, higher CPU frequencies should be allocated to each device as much as possible; combined with energy constraints... Derive the upper bound of the CPU calculation frequency for each device, and substitute the energy expression to obtain the constraint: The results were: This upper bound is determined by the device's available remaining energy, i.e., total energy minus transmission energy consumption, and the scale of the local computing task; in addition, the frequency must also meet hardware capability constraints. Therefore, under the premise of simultaneously satisfying energy and hardware constraints, the objective function is to... The optimal CPU computing frequency should be the smaller of the two values, i.e.: in, The optimal CPU calculation frequency for device k.
8. The method for joint resource optimization of a synesthetic computing fusion system based on federated learning according to claim 5, characterized in that, Step 4.4 specifically involves: Given the device selection variables and the device's CPU computing frequency, the communication beamforming vector and the sensing beamforming matrix are jointly optimized. The joint beamforming optimization subproblem is expressed as follows: Among them, the total system latency , For the selected subset of devices to be scheduled, the objective function is... The goal is to minimize the end-to-end latency determined by the slowest device; this is a non-convex optimization problem. To apply efficient convex optimization tools, the non-convex problem must be approximated. First, auxiliary variables are introduced. Using epigraph transformation to solve the original problem Equivalent transformation to: Despite the equivalent transformation, the constraint non-convexity remains unchanged. Therefore, successive convex approximation is adopted to approximate the optimal solution of the original problem through a series of convex approximations. Convexify the energy constraint: Total energy consumption of equipment middle, and non-convex Related; to decouple, the arithmetic-geometric mean inequality is used to link the constraints. Transform into a topic about and transmission power Second-order cone constraint: Introducing positive semidefinite matrices and Uplink communication rate rewritten in differential form: make: The first term is a concave function, and the second term... It is a convex function; since the negative value of a concave function is convex, therefore It is the difference between concave and convex functions, and the whole is non-convex; using The concavity at point Performing a first-order Taylor expansion at this point, we obtain its upper bound: （29） Substituting (29) into (27), we obtain the rate. A convex lower bound approximation : Introduce the covariance matrix and use the matrix identity: and Rewrite it in difference convex form: In this formula, the first term is a concave function and the second term is a convex function; Similar to handling uplink communication rates, the convex terms in equation (31) are... At the reference point Performing a first-order Taylor expansion at this point, we obtain its upper bound: Substituting into (31), we obtain mutual information. A convex lower bound approximation : in Since is a constant, the following convex optimization problem is obtained: The question The optimal communication beamforming vector and sensing beamforming matrix can be obtained by using convex optimization tools.