Low-overhead multi-point time-frequency positioning method based on federated learning framework

Abstract

The embodiments of the present application relate to the field of wireless communication technology, and in particular to a low-overhead multi-point time-frequency positioning method based on a federated learning framework, comprising: establishing a broadband cognitive network system model and selecting a time-frequency positioning model; a central server initializing the network parameters of the time-frequency positioning model and distributing them to each secondary user; the secondary user locally trains the local time-frequency positioning model based on a local data set, determines the subnet model weight, then extracts the gradient change of each BN layer, sorts based on this, dynamically prunes the local time-frequency positioning model according to the target pruning rate, obtains a model mask, and uploads it to the central server; the central server completes the model using each model mask, aggregates the weights of each subnet model, obtains a global weight, forms a personalized subnet model for each secondary user, and distributes it, thereby achieving low-overhead multi-point time-frequency positioning. This method effectively reduces the communication overhead of the wireless link and significantly improves communication efficiency.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of wireless communication technology, and in particular to a low-overhead multi-point time-frequency positioning method based on a federated learning framework. Background Art

[0002] Spectrum sensing is a key technology in cognitive radio (CR). In cognitive radio, in order to improve spectrum utilization and ensure that secondary users (SUs) can access the frequency band in a timely manner without causing interference to primary users (PUs), it is necessary to frequently perform spectrum sensing on this frequency band. With the increasing demand for broadband spectrum in the development of communications, the demand for spectrum sensing has also shifted from narrowband to broadband. In order to detect multiple channels while adapting to the dynamically changing wireless environment, a broadband spectrum sensing method based on time-frequency positioning has emerged. This method can not only detect the presence of primary users in the frequency band, but also simultaneously detect the time and frequency ranges of multiple signals in the time-frequency spectrum, making it more convenient to implement downstream tasks.

[0003] To expand the sensing range and prevent fluctuations in channel conditions affecting the perception results of a single sensing node, practical scenarios typically employ multiple sensing nodes. This means that multiple sensing nodes collaborate to learn, leveraging available data to improve spectrum sensing accuracy. Traditional centralized learning methods incur significant communication overhead and raise privacy concerns due to the sharing of sensitive user data. Federated learning effectively addresses these issues. Federated learning trains models locally on devices and does not share raw data. Instead, it aggregates the parameters of local models to form a global model, thereby protecting private data and reducing communication costs. Therefore, federated learning is well-suited for distributed spectrum sensing tasks.

[0004] However, in complex and dynamically changing electromagnetic environments, the power, type, and number of signals perceived by SUs vary. Consequently, the datasets used for federated learning training may not be independent and identically distributed (IID). This makes it difficult for the global model trained by federated learning to converge, and can also lead to a decrease in detection performance. The target detection model used in time-frequency localization is complex and parameter-intensive. Frequent interaction with the central server places significant pressure on wireless communication links. Consequently, the problems of data heterogeneity and excessive communication costs during training urgently need to be addressed. Summary of the Invention

[0005] In view of this, the embodiments of the present application propose a low-overhead multi-point time-frequency positioning method based on a federated learning framework, and design a federated learning architecture for heterogeneous data, which effectively reduces the communication overhead of the wireless link, greatly improves the communication efficiency, and thereby improves the perception performance of secondary users in complex electromagnetic environments.

[0006] In order to achieve the above-mentioned purpose, the embodiment of the present application proposes a low-overhead multi-point time-frequency positioning method based on a federated learning framework, comprising the following steps: establishing a broadband cognitive network system model consisting of M primary users, N secondary users and a central server, and selecting a time-frequency positioning model; wherein M and N are both integers greater than 1; the central server initializes the network parameters of the time-frequency positioning model and sends them to each secondary user; the secondary user locally trains the local time-frequency positioning model based on the local data set, determines the updated subnet model weight, and saves the accumulated gradient change; the secondary user extracts the gradient of each BN layer. The gradient change amount is calculated, and each BN layer is sorted based on the gradient change amount. The local time-frequency positioning model is dynamically pruned according to the target pruning rate to obtain the subnet model parameters and model mask, and the model mask and subnet model weight are uploaded to the central server; the central server uses each model mask to complete the global time-frequency positioning model, aggregates the weights of each subnet model, and obtains the aggregated global weight. Based on the aggregated global weight and each model mask, a personalized subnet model for each secondary user is formed and sent to the corresponding secondary user; the secondary user performs time-frequency positioning based on the personalized subnet model, thereby realizing low-overhead multi-point time-frequency positioning.

[0007] In order to achieve the above-mentioned purpose, an embodiment of the present application also proposes an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the low-overhead multi-point time-frequency positioning method based on the federated learning framework as described above.

[0008] To achieve the above objectives, an embodiment of the present application also proposes a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it can implement the low-overhead multi-point time-frequency positioning method based on the federated learning framework as described above.

[0009] The embodiment of the present application proposes a low-overhead multi-point time-frequency positioning method based on a federated learning framework, which creates a broadband spectrum sensing scenario and accurately locates the non-independent and identically distributed problem in distributed time-frequency positioning. It combines the model pruning strategy with federated learning to control the amount of uploaded parameters. When each secondary user performs local training of the time-frequency positioning model, it is necessary to dynamically prune the BN layer. When uploading, the subnet model weights and the model mask that can represent the pruning results are uploaded to the central server together. This design reduces the amount of model parameters that need to be uploaded to the central server and effectively reduces the communication overhead of the wireless link. This is especially important for resource-constrained devices and can effectively reduce the time and cost required for data transmission. In addition, the heterogeneity of data distribution among secondary users is also taken into account. After aggregation, the central server combines the model mask to form a personalized subnet model for each secondary user and sends it down, retaining the personalized characteristics of the model, thereby better adapting to the heterogeneity of local data, effectively improving the prediction accuracy of the time-frequency positioning model, and thus significantly improving the perception performance of secondary users in complex electromagnetic environments.

[0010] In some optional embodiments, the time-frequency positioning model consists of two parts: a feature extraction structure and a prediction structure; the feature extraction structure adopts the Darknet-53 architecture, which consists of a convolutional layer, a residual block, a BN layer, a ReLU activation function layer and a final 1×1 convolutional layer. Residual connections are established between the residual blocks to suppress gradient disappearance; the prediction structure consists of prediction layers added at different depths, located at different layers of the feature extraction structure, to correspond to feature maps of different sizes and detect targets of different sizes.

[0011] In some optional embodiments, in the tth round of communication, the central server randomly selects m secondary users from the secondary users participating in the tth round of communication to form a secondary user set S t ; Where t is an integer greater than 1, m is an integer greater than 1 and less than M; each secondary user divides the local data set into several batches, and the size of each batch is B; in the tth round of communication, S t The secondary users in the algorithm calculate the gradient of the loss function for each batch of local data and perform gradient descent updates, repeating E rounds to determine the updated subnetwork model weights and save the accumulated gradient changes. Repeating E rounds means that the local data of each batch will be used E times for update.

[0012] The updated subnetwork model weight is expressed by the following formula:

[0013]

[0014] in, Indicates S tThe subnet model weight updated by the i-th secondary user in the t-1th round of communication, η is the preset learning rate, g i Indicates S t The back propagation error during training of the i-th secondary user in, Indicates S t The subnetwork model weight updated by the i-th secondary user in the t-th round of communication;

[0015] Saving the cumulative gradient change is achieved through the following formula:

[0016]

[0017] in, Indicates S t The gradient change of the i-th secondary user in the e-th round of gradient descent update, Indicates S t The cumulative gradient change of the i-th secondary user in .

[0018] In some optional embodiments, the secondary user extracts the gradient change of each BN layer, sorts the BN layers based on the gradient change, dynamically prunes the local time-frequency positioning model according to the target pruning rate, and obtains the subnet model parameters and model mask, including: in the tth round of communication, S t The secondary users in the subnet model extract the gradient change of each BN layer of their own subnet model and sort them from large to small according to the gradient change; according to the target pruning rate r target Determine the target parameter size P target , based on P target Dynamically prune the local time-frequency positioning model and set the gradient change less than P target The BN layer is determined as the BN layer that needs to be cut off, and the subnet model parameter P is obtained at the same time. i ; Among them, |P target -P i |≤ξ, ξ is the preset pruning check threshold; establish a vector of size V, V is the total number of BN layers, the elements in the vector represent the BN layers arranged in ascending order, set the value of the element in the vector representing the BN layer to be retained to 1, and set the value of the element in the vector representing the BN layer to be pruned to 0, and obtain the model mask

[0019] In some optional embodiments, the central server uses each model mask to complete the global time-frequency positioning model, aggregates the weights of each subnet model, and obtains the aggregated global weight, which is implemented by the following formula:

[0020]

[0021] Among them, Di represents the local dataset of the i-th secondary user, D j represents the local dataset of the jth secondary user, w t+1 Represents the global weight after aggregation.

[0022] In some optional embodiments, the central server forms a personalized subnet model for each secondary user based on the aggregated global weight and each model mask, which is implemented by the following formula:

[0023]

[0024] Among them, ⊙ represents element-by-element multiplication, represents the personalized sub-network model weight of the i-th secondary user.

[0025] In some optional embodiments, S t After the secondary users in the network obtain the subnet model parameters and the model mask, the method further includes: fine-tuning the pruned subnet model based on the local data set.

[0026] In some optional embodiments, after the central server forms a personalized subnet model for each secondary user based on the aggregated global weights and each model mask, and sends it to the corresponding secondary user, the method further includes: each secondary user completes the received personalized subnet model based on the model mask, determines whether the completed personalized subnet model meets the convergence conditions, and ends local training if it is determined that the convergence conditions are met; each secondary user performs time-frequency positioning based on the corresponding completed personalized subnet model, thereby achieving low-overhead multi-point time-frequency positioning. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the related technologies, the following is a brief introduction to the drawings required for use in the embodiments of the present application or the description of the related technologies. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0028] Figure 1 This is a flowchart of a low-overhead multi-point time-frequency positioning method based on a federated learning framework provided in one embodiment of the present application;

[0029] Figure 2 is a scene diagram of a broadband cognitive network system model provided in one embodiment of the present application;

[0030] Figure 3 is a schematic diagram of model pruning classification provided in one embodiment of the present application;

[0031] Figure 4 is a flowchart of model pruning provided in one embodiment of the present application;

[0032] Figure 5 is a schematic diagram of federated learning provided in one embodiment of the present application;

[0033] Figure 6 is a schematic diagram of a multi-model performance comparison provided in one embodiment of the present application;

[0034] Figure 7 It is a structural diagram of an electronic device provided in another embodiment of the present application. DETAILED DESCRIPTION

[0035] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, each embodiment of the present application will be described in detail below with reference to the accompanying drawings. However, it will be understood by those skilled in the art that in each embodiment of the present application, many technical details are proposed to enable the reader to better understand the present application. However, even without these technical details and various changes and modifications based on the following embodiments, the technical solutions claimed in the present application can be implemented. The division of the following embodiments is for convenience of description and should not constitute any limitation on the specific implementation of the present application. The various embodiments can be combined and referenced with each other under the premise of no contradiction.

[0036] Spectrum sensing is a key technology in cognitive radio. With the development of broadband technology and the increasing flexibility of bandwidth usage, spectrum sensing technology needs to monitor spectrum resource utilization across a wide bandwidth. Consequently, wideband wireless broadband technology has attracted widespread attention. Numerous machine learning-based methods, particularly those based on time-frequency localization, have emerged. These methods can identify frequency hopping patterns, signal widths, and signal dwell times without prior knowledge, facilitating downstream spectrum management tasks. Consequently, wideband spectrum sensing based on time-frequency localization has also attracted the attention of researchers.

[0037] In real-world scenarios, multiple sensor nodes often collaborate to learn and leverage available data to improve spectrum sensing accuracy. However, traditional centralized learning methods incur significant communication overhead and raise privacy concerns due to the sharing of sensitive user data. Federated learning effectively addresses these issues. Federated learning trains models locally on devices without sharing raw data. It aggregates local model parameters to obtain a global model, thereby protecting data privacy and reducing communication costs. Therefore, federated learning is well-suited for distributed spectrum sensing tasks.

[0038] Current research focuses on narrowband spectrum sensing methods within a federated learning framework. Troglia et al. proposed a distributed sensing architecture for in-situ detection in a citizen broadband radio service system, demonstrating that federated learning can improve detection performance. Chen et al. first introduced federated learning into collaborative spectrum, proposed a complete process for spectrum sensing within a federated learning framework, and demonstrated the effectiveness of the federated learning framework for collaborative spectrum sensing performance. Miao et al. used the k-means algorithm to cluster SUs, which then participated in federated learning to sense 5G downlink signals. Zhang Y et al. pointed out that the received signal strength values ​​measured by SUs at different locations represent different characteristics of the PU activity pattern. In the aggregation phase, the features of the same PU samples are aligned, allowing the global model to learn the complete characteristics of the PU activity pattern to improve the perception effect. Based on this, Zhang Z et al. proposed a truncated longitudinal federated learning framework to achieve fast convergence under limited communication and computing resources.

[0039] The above studies simply demonstrate that federated learning reduces communication costs compared to other distributed architectures, but do not consider the communication costs of wireless links in federated learning. However, it is necessary to consider, in combination with model size and communication frequency, whether the wireless link can bear the communication overhead of model upload. All of these studies leverage the advantages of federated learning and combine it with multi-point spectrum sensing to improve spectrum sensing efficiency. However, in complex and changing electromagnetic environments, different sensing nodes are located in different locations, and the data they sense can vary significantly. The data collected by these sensing nodes is not independent and identically distributed, and data heterogeneity can significantly negatively impact federated learning algorithms. Zhang Y, Zhang Z, et al. considered the data heterogeneity of SUs with different power distributions in narrowband spectrum sensing. However, the same solution is difficult to alleviate the data heterogeneity problem in wideband spectrum sensing. This is because wideband spectrum sensing involves complex data heterogeneity, and time-frequency localization methods require complex features such as shape, size, position, and texture of time-frequency blocks. Aligning all these features would significantly increase the workload.

[0040] In the case of heterogeneous data, we found that unstable global model performance stems from the fact that the parameters sent to each SU are identical. Without adjusting the global model's structure and parameters, the model cannot adapt to local data. Model pruning allows each SU to obtain a personalized model adapted to its local data, mitigating the impact of data heterogeneity. Model pruning also removes redundant model weights and connections, effectively reducing the number of parameters uploaded to the central server and further reducing model communication overhead.

[0041] Based on this, an embodiment of the present application proposes a low-overhead multi-point time-frequency positioning method based on a federated learning framework. The following is a specific description of the implementation details of a low-overhead multi-point time-frequency positioning method based on a federated learning framework proposed in this embodiment. The following content is only the implementation details provided for easy understanding and is not necessary for the implementation of this solution.

[0042] The specific process of the low-overhead multi-point time-frequency positioning method based on the federated learning framework proposed in this embodiment can be as follows: Figure 1 As shown, including:

[0043] Step 101: Establish a broadband cognitive network system model consisting of M primary users, N secondary users, and a central server, and select a time-frequency positioning model.

[0044] In the specific implementation, the scenarios of the broadband cognitive network system model are as follows: Figure 2 As shown in the figure, the broadband cognitive network system model consists of M primary users, N secondary users, and a central server, where M and N are both integers greater than 1. Primary users can dynamically select the center frequency and bandwidth in the broadband spectrum based on service requirements or interference prevention. Secondary users are responsible for using machine learning detection heads to perceive the spectrum occupancy in the broadband and upload local models to the central server after a certain number of rounds of local training.

[0045] During a spectrum sensing period, the primary user independently determines its bandwidth and center frequency. At the same time, each secondary user collects K signal samples. The signal of a primary user perceived by a secondary user can be modeled as follows:

[0046] x n (k)=h*s m (k)+ε(k);

[0047] 0<n≤N;

[0048] 0<m≤M;

[0049] Among them, x n (k) represents the kth signal sample perceived by the nth secondary user, s m (k) represents the kth signal sample sent by the mth primary user, h represents the channel gain between the primary user and the secondary user, and ε(k) represents the noise in the sample. Based on this formula, we can further explain the phenomenon of data heterogeneity. When SU ​​perceives, s m(k) is controlled by factors such as signal system, center frequency, signal bandwidth, and transmission power. At the same time, the geographical location and environment of the secondary user and the primary user in the electromagnetic environment will affect the degree of signal path loss and multipath effect, resulting in different channel fading and ultimately affecting the size of h. Therefore, the signal x that the secondary user can perceive is n (k) Affected by multiple factors, different secondary users experiencing the same primary user's signal will produce different signal data distributions, resulting in non-independent and identically distributed data between secondary users. Therefore, heterogeneous data is inherently present in this scenario, and its impact needs to be considered.

[0050] After receiving the signal, we also need to process the signal and extract its two-dimensional information in the time domain and frequency domain. Among them, short-time Fourier transform (STFT) is a commonly used time-frequency analysis method that can provide time and frequency information at the same time. STFT divides the signal into multiple short time periods by using a sliding window and performs Fourier transform on each short time period to obtain the spectrum information of the signal in different time periods. Therefore, we assume that the sliding window function used is w(k), L represents the time index of each window function, and I represents the number of Fourier transform points of the short time period. The result of the signal received by the secondary user after STFT is as follows:

[0051]

[0052] Wherein, S(i, k) represents the result obtained after STFT of the k-th signal sample perceived by the i-th secondary user.

[0053] In the STFT results, the horizontal axis does not represent time, and the vertical axis represents frequency. The time-frequency block obtained after visualization can clearly indicate the usage of time and frequency, display the time-frequency characteristics of the signal, and facilitate time-frequency positioning tasks.

[0054] Time-frequency localization requires locating specific time-frequency blocks in the time-frequency spectrum obtained after STFT. This task can be compared to object detection in an image, so the time-frequency localization model can be adapted from an object detection model. Given the need for rapid results in wideband time-frequency localization, the YOLO model, as a single-stage detection method, directly outputs targets. Its efficient algorithm and comprehensive model structure make it suitable for time-frequency localization. YOLO typically performs two key steps in time-frequency localization: the backbone network extracts positional and semantic information from the input time-frequency spectrum, followed by object classification and bounding box regression, resulting in a multidimensional vector containing the bounding box location and signal category information. Once the time-frequency localization model is established, it needs to be deployed within a federated learning framework. Inspired by the combination of federated learning and narrowband time-frequency localization, we combine wideband time-frequency localization with federated learning to efficiently train a time-frequency localization model while protecting data privacy and reducing overhead.

[0055] In one example, a time-frequency localization model consists of a feature extraction structure and a prediction structure. The feature extraction structure uses the Darknet-53 architecture, consisting of convolutional layers, residual blocks, batch normalization layers, ReLU activation layers, and a final 1×1 convolutional layer. Residual connections are established between residual blocks to prevent vanishing gradients. The prediction structure consists of prediction layers added at different depths, located at different layers of the feature extraction structure, to correspond to feature maps of different sizes and detect objects of different sizes.

[0056] Step 102: The central server initializes the network parameters of the time-frequency positioning model and sends them to each secondary user.

[0057] In a specific implementation, the central server initializes the network parameter W0 of the time-frequency positioning model and sends it to all secondary users in the broadband cognitive network system model.

[0058] In an example, in the tth round of communication, the central server randomly selects m secondary users from the secondary users participating in the tth round of communication to form a secondary user set S t , where t is an integer greater than 1, and m is an integer greater than 1 and less than M. S t Each secondary user in is the secondary user participating in this round of federated learning.

[0059] In step 103 , the secondary user performs local training on the local time-frequency positioning model based on the local data set, determines the updated sub-network model weights, and saves the accumulated gradient changes.

[0060] In step 104, the secondary user extracts the gradient change of each BN layer, sorts the BN layers based on the gradient change, dynamically prunes the local time-frequency positioning model according to the target pruning rate, obtains the subnet model parameters and model mask, and uploads the model mask and subnet model weight to the central server.

[0061] In the specific implementation, the secondary users conduct local training on the local time-frequency positioning model in the federated learning. Each secondary user sends the local dataset (using D i represents the local data set of the i-th secondary user) is divided into several batches, each of which has a size of B. In the t-th round of communication, S t Each secondary user in the S calculates the gradient of the loss function for each batch of local data and performs gradient descent update, repeating E rounds, that is, each batch of local data will be used E times for update, determine the updated subnetwork model weight, and save the cumulative gradient change. t The secondary users in the subnet model extract the gradient change of each BN layer of their own subnet model and sort them from large to small according to the gradient change. target Determine the target parameter size P target , based on P target Dynamically prune the local time-frequency positioning model and set the gradient change less than P target The BN layer is determined as the BN layer that needs to be cut off, and the subnet model parameter P is obtained at the same time. i Among them, |P target -P i |≤ξ, ξ is the preset pruning check threshold. Then, a vector of size V is established, where V is the total number of BN layers. The elements in the vector represent the BN layers arranged in ascending order. The values ​​of the elements in the vector representing the BN layers to be retained are set to 1, and the values ​​of the elements in the vector representing the BN layers to be pruned are set to 0 to obtain the model mask.

[0062] In one example, the updated subnetwork model weight is expressed by the following formula:

[0063]

[0064] in, Indicates S t The subnet model weight updated by the i-th secondary user in the t-1th round of communication, η is the preset learning rate, g i Indicates S t The back propagation error during training of the i-th secondary user in, Indicates S t The subnetwork model weights updated by the i-th secondary user in the t-th round of communication.

[0065] In one example, the accumulated gradient change is saved using the following formula:

[0066]

[0067] in, Indicates S t The gradient change of the i-th secondary user in the e-th round of gradient descent update, Indicates S t The cumulative gradient change of the i-th secondary user in .

[0068] Given the limited overhead of wireless communication links, the need to reduce model parameter count is crucial. Model pruning, a key component of model compression, is our preferred approach. This reduces the number of metamodel parameters and computational complexity, while ensuring that submodel losses are manageable, helping to alleviate communication pressure and improve computational efficiency. Furthermore, pruning helps identify the model that best suits the local dataset, removing redundant parameters and connections, and personalizing the model to adapt to heterogeneous data.

[0069] like Figure 3 As shown, model pruning is divided into structured pruning and unstructured pruning. Unstructured pruning is usually performed at the parameter level, directly pruning individual weights in the model without considering the location of the weight or the influence of other weights, and ignoring the connections between weights. Therefore, the model can be very sparse, but it also destroys the structure of the original model and requires specific hardware support to achieve improved computational efficiency. Structured pruning is usually performed by pruning entire channels or filters, and does not simply prune scattered weights. Although the previous eigenvalue and the next eigenvalue will change accordingly, the internal structure of the channel is not destroyed. Structured pruning cannot achieve the model compression level of unstructured pruning, but it can be better used in conjunction with modern hardware acceleration optimization technologies to achieve improved computational efficiency. Therefore, this embodiment adopts a pruning strategy of structured pruning channels to obtain a sub-network of the local model and reduce the amount of uploaded model parameters.

[0070] In this embodiment, the personalized pruning operation will be carried out throughout the entire local training process of the secondary user to ensure that the parameters uploaded to the central server each time are the sparse model after pruning. Therefore, we have added personalized pruning operations to the normal training, so that the model becomes gradually streamlined during the model training process. The standard process is as follows: Figure 4 As shown, the following focuses on the training + importance sorting, mask acquisition and fine-tuning process.

[0071] The first is the training + importance sorting process. The training with the introduction of pruning operations is different from the normal local training process. The difference between the two is that the process of model sensitivity analysis is added, which requires analyzing specific weights during the training process to determine the importance. Compared with sorting by weight size, the pruning method of this application abandons this direct and crude method, and instead uses a more sophisticated sorting method based on the amount of change in weight gradients. This method can be dynamically adjusted during the training process and can adapt to the needs of different training stages as the gradient changes.

[0072] This application uses the change in weight gradient as the ranking criterion for importance. The definitions of gradient and gradient accumulation are shown in the following formula:

[0073]

[0074]

[0075] Among them, w i is the i-th element in the model parameters, T is the number of iterations, w (t) is the parameter value at iteration t. The gradient itself represents the local rate of change of the loss function with respect to the model parameters. The magnitude of the gradient reflects the local sensitivity of the loss function to the model parameters. A large gradient for a parameter indicates that the change in the loss function is significant within the parameter's neighborhood, indicating that the parameter has a significant impact on the model output. During local training, a large cumulative update of a parameter indicates that it plays a key role in model training.

[0076] The pruning involved in this application is an operation performed on the entire channel, so there is no need to observe the gradient changes of each weight at a small scale. Since the convolution layer and the BN layer usually appear in pairs in the structure of the model, and the output of the convolution layer corresponds to the input of the BN layer, this application seeks a breakthrough in the importance ranking of channel pruning from the BN layer. The BN layer pair is a normalization operation on the data of the convolution layer. During the training process, each feature channel w i The input data z in Normalize the data so that its output is the mean z out Close to 0, the standard deviation is close to 1. The process is:

[0077] First, the mean and variance of all data in the feature channel are calculated to obtain μ z and Then the input data z in Normalize it and take its mean μ z Move to 0 and scale the variance to 1, as follows:

[0078]

[0079] Here, ε is a small positive number used to prevent the denominator from being zero.

[0080]

[0081] Finally, the BN layer introduces two parameters, γ and β, which can be changed during training. These are the scaling factor and the offset factor, respectively, which are used to adjust the normalized values ​​so that the model can restore its learning ability. γ is responsible for scaling the normalized data. If the value of γ is close to 1, it has little effect on the output of the channel. If the value of γ is close to 0, it means that the output of the channel is significantly reduced and may contribute little to the final model output. If the gradient of y changes significantly over multiple iterations, it may indicate that the model is actively adjusting to adapt to the training data. Therefore, we use the weight changes of the BN layer as a ranking of importance. By resetting the weights of the BN layer to 0, it is equivalent to turning off the output of the channel, and it has almost no contribution to the input of subsequent layers.

[0082] In summary, in addition to the normal training process, in this training phase, by sorting the weight gradient changes in the BN layer and finding the weight pruning threshold according to the target pruning rate, the weights of unimportant models are forced to 0, and the unimportant channels in the model are sparsed. This method allows for dynamic adjustment of which channels are important and which are unimportant during training, which provides flexibility for the model, enabling it to adjust its behavior according to training data and task requirements.

[0083] Next, we need to obtain the mask. During sparse training, the importance of each channel needs to be evaluated. This criterion typically compares the channel weights to a threshold of importance channel weights determined by the target pruning rate, selecting those channels that are highly important and removing those that are less important. This process is achieved by multiplying the weights by a mask. The mask is a tensor of the same shape as the weights, containing either 0 or 1 values. The mask determines which weights and biases should be retained and which should be set to zero. This multiplication results in the pruning of certain channels in the model, thereby reducing the model size while minimizing performance.

[0084] Finally, fine-tuning, S t After obtaining the subnet model parameters and model mask, the secondary user in

[15] needs to fine-tune the pruned subnet model based on the local dataset. This is because channel pruning may slightly degrade model performance. Fine-tuning aims to restore performance while maintaining model compression. This process is repeated until the ideal compression model is achieved.

[0085] In step 105, the central server uses the model masks to complete the global time-frequency positioning model, aggregates the weights of the subnet models, obtains the aggregated global weight, forms the personalized subnet model of each secondary user based on the aggregated global weight and the model masks, and sends it to the corresponding secondary user.

[0086] In the specific implementation, Figure 5 As shown in the figure, after each secondary user completes local training and uploads the model mask and subnet model weights, the central server begins aggregate training. First, the global time-frequency positioning model is completed using each model mask. The subnet model weights are aggregated to obtain the aggregated global weights. Then, based on the aggregated global weights and each model mask, a personalized subnet model is formed for each secondary user and distributed to the corresponding secondary user.

[0087] In one example, the central server uses the masks of each model to complete the global time-frequency positioning model, aggregates the weights of each subnet model, and obtains the aggregated global weight, which is achieved through the following formula:

[0088]

[0089] Among them, D i represents the local dataset of the i-th secondary user, D j represents the local dataset of the jth secondary user, w t+1 Represents the global weight after aggregation.

[0090] In one example, the central server forms a personalized subnet model for each secondary user based on the aggregated global weight and each model mask, which is implemented by the following formula:

[0091]

[0092] Among them, ⊙ represents element-by-element multiplication, represents the personalized sub-network model weight of the i-th secondary user.

[0093] In one example, after the central server forms a personalized subnet model for each secondary user based on the aggregated global weights and each model mask and sends it to the corresponding secondary user, each secondary user completes the received personalized subnet model based on the model mask and determines whether the completed personalized subnet model meets the convergence conditions. If it is determined that the convergence conditions are met, the local training ends.

[0094] In step 106, the secondary user performs time-frequency positioning based on the personalized subnet model, thereby achieving low-overhead multi-point time-frequency positioning.

[0095] In a specific implementation, each secondary user in the broadband cognitive network system model can perform time-frequency positioning based on their corresponding completed personalized subnet model, thereby achieving low-overhead multi-point time-frequency positioning.

[0096] This embodiment creates a broadband spectrum sensing scenario, accurately locates the non-independent and identically distributed problem existing in distributed time-frequency positioning, combines the model pruning strategy with federated learning to control the amount of uploaded parameters, and when each secondary user performs local training of the time-frequency positioning model, it is necessary to dynamically prune the BN layer. When uploading, the subnet model weights and the model mask that can represent the pruning results are uploaded to the central server together. This design reduces the amount of model parameters that need to be uploaded to the central server and effectively reduces the communication overhead of the wireless link. This is especially important for resource-constrained devices and can effectively reduce the time and cost required for data transmission. In addition, the heterogeneity of data distribution among secondary users is also taken into account. After aggregation, the central server combines the model mask to form a personalized subnet model for each secondary user and sends it down, retaining the personalized characteristics of the model, thereby better adapting to the heterogeneity of local data, effectively improving the prediction accuracy of the time-frequency positioning model, and significantly improving the perception performance of secondary users in complex electromagnetic environments.

[0097] The step division of the above various methods is only for the purpose of clear description. During implementation, they can be combined into one step or some steps can be split and decomposed into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this application; adding insignificant modifications or introducing insignificant designs to the algorithm or process without changing the core design of the algorithm and process are all within the scope of protection of this application.

[0098] In one embodiment, we conducted simulation experiments using a public signal dataset published by Nguyen. This dataset contains five different signal categories, including Bluetooth, Zigbee, WiFi, WMIC, and Lightbridge. The signal processing was consistent with the requirements, and time-frequency analysis yielded corresponding signal time-frequency maps. Labels indicated the signal category and coordinates within the map. This dataset simulated real-world signal data under varying signal-to-noise ratios by adjusting parameters such as transmit power and burst duration, providing sufficient data for segmenting heterogeneous SUs.

[0099] To ensure that the local data of SUs is not independent and identically distributed, the dataset is divided into common and non-common parts according to category. The common part represents samples of a category that are present in every SU, while the non-common part represents samples that are missing from some SUs. We define Wi-Fi, Bluetooth, and any overlap between them as the common part. The specific dataset divisions are shown in Table 1:

[0100] Table 1: Dataset division

[0101]

[0102] Each SU saves the received 375 image spectrograms locally, where the local training set and the local test set are divided into a ratio of 4:1.

[0103] To demonstrate the advantages of the federated learning architecture and the effectiveness of introducing pruning-based federated learning into the federated architecture, we conducted single-point model training, classic federated learning, and pruning-based federated learning model training. The following are the parameter settings for the training process:

[0104] Classic federated training: A federated learning system is set up with three clients participating in training. Each client participates in training in each communication round, and the model is iterated five times on the local dataset. The maximum number of communication rounds between the client and the server is 100. Client-side local training uses the Adam optimizer with a learning rate of 0.001 and a batch size of 16 randomly selected from all datasets.

[0105] Single-node model training: The training parameters of each single node are the same as those of each client in classic federated training.

[0106] Pruning-based federated training (this application): Set up a federated learning system with 3 clients participating in training, add structured pruning operations, set different target pruning rates, and ensure that at least 1% of channels are retained in each layer. The remaining parameters are the same as those of classic federated learning training.

[0107] The performance comparison of these three methods is as follows: Figure 6 As shown, from Figure 6 The performance of the single-node model, the federated learning model, and the federated model after pruning on the local dataset is shown. The federated learning model performs particularly well. This demonstrates that the federated learning client is able to effectively learn the global distribution characteristics of the data, enhancing the model's perception capabilities. Furthermore, the pruned federated model generally exhibits improved performance compared to standard federated learning. This means that the federated model with pruning can better adapt to the local dataset and possess stronger personalization capabilities. This demonstrates that pruning can be used within the federated learning framework to enhance model personalization, resulting in a personalized model adapted to the local dataset and improving the poor performance of the global model obtained through federated learning on heterogeneous datasets. This demonstrates the effectiveness of multi-node collaborative training and structured pruning. The addition of federation improves the ability to perceive the global data distribution, and the pruning operation enhances the recognition performance of the model on local data, building on this global data distribution awareness.

[0108] Another difficult problem solved by this application is the limited communication overhead on the wireless link. Through structured pruning, the amount of model parameters uploaded and downloaded can be controlled, further reducing the communication overhead generated during the collaborative training process.

[0109] Table 2: Model parameter changes with a target pruning rate of 30%

[0110]

[0111] Table 2 shows the change in overhead through specific data. The parameter ratio is reduced by 31.45% under the control of the target pruning rate. Therefore, the model pruning method introduced in this application not only improves performance, but also effectively reduces overhead, removes redundant model parameters and connections, and facilitates efficient communication in resource-limited wireless link situations.

[0112] Table 2 shows that the channel removal rate is different from the target pruning rate. This is because different convolutional layers have varying channel sizes and parameter counts. Pruning smaller feature maps results in minimal model compression, but pruning larger feature maps results in significant model compression. Therefore, the relationship between the pruning rate and model size is not linear, but rather positively correlated. Increasing the pruning rate will inevitably reduce the number of model parameters, but the extent of this change is uncertain.

[0113] Another embodiment of the present application provides an electronic device, the specific structure of which can be as follows: Figure 7 As shown, it includes: at least one processor 201; and a memory 202 that is communicatively connected to the at least one processor 201; wherein the memory 202 stores instructions that can be executed by the at least one processor 201, and the instructions are executed by the at least one processor 201 to enable the at least one processor 201 to execute a low-overhead multi-point time-frequency positioning method based on a federated learning framework as described in the above-mentioned method embodiments.

[0114] The memory and processor are connected using a bus, which can include any number of interconnected buses and bridges. The bus connects one or more processors and various circuits in the memory. The bus can also connect various other circuits such as peripheral devices, voltage regulators, and power management circuits. These are all well known in the art and therefore will not be described further herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be a single component or multiple components, such as multiple receivers and transmitters, providing a unit for communicating with various other devices on a transmission medium. Data processed by the processor is transmitted over a wireless medium via an antenna. Furthermore, the antenna receives data and transmits it to the processor.

[0115] The processor is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory can be used to store data used by the processor when performing operations.

[0116] Another embodiment of the present application proposes a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it can implement a low-overhead multi-point time-frequency positioning method based on a federated learning framework as described in the above method embodiment.

[0117] That is, those skilled in the art will understand that all or part of the steps in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a program, which is stored in a storage medium and includes a number of instructions for causing a device (which may be a single-chip microcomputer or chip, etc.) or a processor to execute all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes, but is not limited to, various media that can store program code, such as read-only memory (ROM), random access memory (RAM), USB flash drive, mobile hard disk, magnetic disk or optical disk.

[0118] Those skilled in the art will appreciate that the above embodiments are specific embodiments for implementing the present application, and that in actual applications, various changes may be made thereto in form and detail without departing from the spirit and scope of the present application.

Claims

1. A low-overhead multi-point time-frequency positioning method based on a federated learning framework, characterized in that: include: A broadband cognitive network system model consisting of M primary users, N secondary users, and a central server is established, and a time-frequency positioning model is selected; where M and N are both integers greater than 1; The central server initializes the network parameters of the time-frequency positioning model and sends them to each secondary user; The secondary user performs local training on the local time-frequency positioning model based on the local dataset, determines the updated subnetwork model weights, and saves the accumulated gradient changes. The secondary user extracts the gradient change of each BN layer, sorts the BN layers based on the gradient change, dynamically prunes the local time-frequency positioning model according to the target pruning rate, obtains the subnet model parameters and model mask, and uploads the model mask and subnet model weight to the central server; The central server uses the masks of each model to complete the global time-frequency positioning model, aggregates the weights of each sub-network model, and obtains the aggregated global weight. Based on the aggregated global weight and each model mask, it forms a personalized sub-network model for each secondary user and sends it to the corresponding secondary user. Secondary users perform time-frequency positioning based on personalized subnet models, thereby achieving low-overhead multi-point time-frequency positioning.

2. The low-overhead multi-point time-frequency positioning method based on the federated learning framework according to claim 1, characterized in that: The time-frequency localization model consists of two parts: feature extraction structure and prediction structure; The feature extraction structure uses the Darknet-53 architecture, which consists of a convolutional layer, a residual block, a batch normalization layer, a ReLU activation function layer, and a final 1×1 convolutional layer. Residual connections are established between residual blocks to suppress gradient disappearance. The prediction structure consists of prediction layers added at different depths, located at different layers of the feature extraction structure to correspond to feature maps of different sizes and detect objects of different sizes.

3. The low-overhead multi-point time-frequency positioning method based on the federated learning framework according to claim 2, characterized in that: In the tth round of communication, the central server randomly selects m secondary users from the secondary users participating in the tth round of communication to form the secondary user set S t ; Wherein, t is an integer greater than 1, and m is an integer greater than 1 and less than M; Each secondary user divides the local dataset into several batches, and the size of each batch is B; In the tth round of communication, S t The secondary users in the algorithm calculate the gradient of the loss function for each batch of local data and perform gradient descent updates, repeating E rounds to determine the updated subnetwork model weights and save the accumulated gradient changes. Repeating E rounds means that the local data of each batch will be used E times for update. The updated subnetwork model weight is expressed by the following formula: in, Indicates S t The subnet model weight updated by the i-th secondary user in the t-1th round of communication, η is the preset learning rate, g i Indicates S t The back propagation error during training of the i-th secondary user in, Indicates S t The subnetwork model weight updated by the i-th secondary user in the t-th round of communication; Saving the cumulative gradient change is achieved through the following formula: in, Indicates S t The gradient change of the i-th secondary user in the e-th round of gradient descent update, Indicates S t The cumulative gradient change of the i-th secondary user in .

4. The low-overhead multi-point time-frequency positioning method based on the federated learning framework according to claim 3, characterized in that: The secondary user extracts the gradient change of each BN layer, sorts the BN layers based on the gradient change, dynamically prunes the local time-frequency positioning model according to the target pruning rate, and obtains the subnet model parameters and model mask, including: In the tth round of communication, S t The secondary users in extract the gradient change of each BN layer of their own sub-network model and sort them from large to small according to the gradient change; According to the target pruning rate r target Determine the target parameter size P target , based on P target Dynamically prune the local time-frequency positioning model and set the gradient change less than P target The BN layer is determined as the BN layer that needs to be cut off, and the subnet model parameter P is obtained at the same time. i ; Among them, |P target -P i |≤ξ, ξ is the preset pruning check threshold; Create a vector of size V, where V is the total number of BN layers. The elements in the vector represent the BN layers arranged in ascending order. Set the values ​​of the elements in the vector that represent the BN layers to be retained to 1, and set the values ​​of the elements in the vector that represent the BN layers to be cut to 0 to obtain the model mask.

5. The low-overhead multi-point time-frequency positioning method based on the federated learning framework according to claim 4, characterized in that: The central server uses the masks of each model to complete the global time-frequency positioning model, aggregates the weights of each subnet model, and obtains the aggregated global weight, which is achieved through the following formula: Among them, D i represents the local dataset of the i-th secondary user, D j represents the local dataset of the jth secondary user, w t+1 Represents the global weight after aggregation.

6. The low-overhead multi-point time-frequency positioning method based on the federated learning framework according to claim 5, characterized in that: The central server forms a personalized subnet model for each secondary user based on the aggregated global weight and each model mask, which is achieved through the following formula: Among them, ⊙ represents element-by-element multiplication, represents the personalized sub-network model weight of the i-th secondary user.

7. The low-overhead multi-point time-frequency positioning method based on a federated learning framework according to any one of claims 3 to 6, characterized in that: S t After the secondary users in the network obtain the subnet model parameters and the model mask, the method further includes: fine-tuning the pruned subnet model based on the local data set.

8. The low-overhead multi-point time-frequency positioning method based on a federated learning framework according to any one of claims 1 to 6, characterized in that: After the central server forms a personalized subnet model for each secondary user based on the aggregated global weight and each model mask, and sends the model to the corresponding secondary user, the method further includes: Each secondary user completes the received personalized subnet model based on the model mask and determines whether the completed personalized subnet model meets the convergence conditions. If it is determined that the convergence conditions are met, the local training ends. Each secondary user performs time-frequency positioning based on its corresponding completed personalized subnet model, thereby achieving low-overhead multi-point time-frequency positioning.

9. An electronic device, characterized in that: include: at least one processor; and, a memory communicatively coupled to the at least one processor; In which, the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute a low-overhead multi-point time-frequency positioning method based on a federated learning framework as described in any one of claims 1 to 8.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, it can implement a low-overhead multi-point time-frequency positioning method based on a federated learning framework as described in any one of claims 1 to 8.

Images