Wireless energy transfer methods, apparatus, devices, and readable storage media

Abstract

This application provides a wireless power transmission method, apparatus, device, and readable storage medium. This application allows for the continuous transmission of wireless power to various users, enabling them to train target models locally and transmit already trained target models. It can also determine whether the instantaneous channel state information of the wireless link is known, and adjust the wireless power resource allocation strategy for each user based on this information. If the instantaneous channel state information is known, a resource allocation strategy based on a deep Q-learning network and a greedy algorithm can be used to optimize the wireless power resource allocation strategy for each user. This effectively avoids the problem of users participating in federated learning being forced to interrupt federated learning due to excessive power consumption or long execution latency, thus preventing the upload of model parameters.

Description

Technical Field

[0001] This application relates to the field of energy transmission technology, and in particular to a wireless energy transmission method, apparatus, device and readable storage medium. Background Technology

[0002] The construction of a digital power grid is a process of digitizing, intelligentizing, and internetizing the traditional power grid. Digital transformation of the traditional power grid requires building a corresponding digital twin power grid. This involves using advanced digital technology platforms to form powerful "computing power" through "computing ability + data + models + algorithms." Relying on the Internet of Things and the Internet, this connects all aspects of the power grid, including perception, analysis, decision-making, and business operations. This empowers power grid companies with superior perception capabilities, informed decision-making abilities, and rapid execution capabilities, extending the boundaries of the digital power grid from the traditional power grid to all aspects of society. It transforms the management, operation, and service models of the traditional power grid, driving the widespread allocation of energy, capital, logistics, business, and talent flows in related industries. By using "electricity + computing power," it promotes the energy revolution and the construction of a new energy system, contributing to the modernization of the national economic system and building a new, inherently secure digital power grid system.

[0003] The rise of artificial intelligence has provided new perspectives and ideas for the development of digital power grids. With the rapid development of AI technology, deep learning has shown enormous potential in fields such as computer vision, signal processing, and wireless communication. However, centralized deep learning requires collecting large amounts of data from distributed users and uploading it to a central server to support centralized training, which poses a serious risk of privacy breaches. Consequently, more and more users are unwilling to share their private data, leading to data silos. To address this issue, the concept of federated learning has been proposed, aiming to promote decentralized intelligence without compromising privacy. In a federated learning framework, users can collaborate with a federated learning server to train a global model by uploading model parameters instead of their private raw data. This ensures the security of private data while reducing communication overhead. However, energy consumption in federated learning is a critical issue for wireless networks, including federated learning networks, as it determines how long users can operate within the network. The number of users participating in federated learning directly affects the performance of the power grid system. How to address the issue of users being forced to withdraw from federated learning due to energy consumption problems has been a persistent concern. Summary of the Invention

[0004] This application aims to address at least one of the aforementioned technical deficiencies. In view of this, this application provides a wireless power transmission method, apparatus, device, and readable storage medium to solve the technical deficiency in the prior art where users are forced to withdraw from federated learning due to unresolved energy consumption issues.

[0005] A wireless power transfer method, comprising:

[0006] Continuously transmit wireless power to each user so that each user can train the target model locally and transmit the locally trained target model.

[0007] Determine whether instantaneous channel state information or statistical channel state information of the wireless link in the energy transmission system has been obtained;

[0008] If the instantaneous channel state information of the wireless link in the energy transmission system is known, a preset resource allocation strategy based on a deep Q-learning network and a greedy algorithm is adopted to optimize the wireless energy resource allocation strategy for each user.

[0009] Preferably, the method further includes:

[0010] If only the statistical channel state information of the wireless links in the energy transmission system is known, a resource allocation strategy based on a connected deep Q-learning network and the golden section method is adopted to optimize the wireless energy resource allocation strategy for each user.

[0011] Preferably, the process of creating the resource allocation strategy based on deep Q-learning networks and greedy algorithms includes:

[0012] Dynamically update the wireless bandwidth obtained by each user from the hybrid access point of the energy transmission system;

[0013] Update the radio frequency power corresponding to the wireless bandwidth obtained by each user from the hybrid access point;

[0014] Based on the updated wireless bandwidth and RF power obtained by each user from the hybrid access point, the wireless charging time of each user at the hybrid access point is updated until the loss function of the preset first resource allocation network model converges, thereby obtaining the resource allocation strategy of the energy transmission system based on a deep Q-learning network and a greedy algorithm. The preset first resource allocation network model is trained using the wireless bandwidth and corresponding RF power obtained by each user from the hybrid access point and the wireless charging time of each user at the hybrid access point as training samples, and the resource allocation strategy corresponding to the wireless bandwidth and corresponding RF power obtained by each user from the hybrid access point and the wireless charging time of each user at the hybrid access point as sample labels.

[0015] Preferably, the process of creating the resource allocation strategy based on the connected deep Q-learning network and the golden section method includes:

[0016] Dynamically update the wireless bandwidth obtained by each user from the hybrid access point of the energy transmission system;

[0017] Update the radio frequency power corresponding to the wireless bandwidth obtained by each user from the hybrid access point;

[0018] Based on the updated wireless bandwidth and corresponding RF power obtained by each user from the hybrid access point, the wireless charging time of each user at the hybrid access point is updated until the loss function of the preset second resource allocation model network converges, thereby obtaining the resource allocation strategy of the energy transmission system based on the connected deep Q-learning network and the golden section method. The preset second resource allocation network model is trained using the wireless bandwidth and corresponding RF power obtained by each user from the hybrid access point and the wireless charging time of each user at the hybrid access point as training samples, and the resource allocation strategy corresponding to the wireless bandwidth and corresponding RF power obtained by each user from the hybrid access point and the wireless charging time of each user at the hybrid access point as sample labels.

[0019] Preferably, the continuous transmission of wireless power to each user includes:

[0020] The transmission time for wireless power to each user is divided into several time slots of equal span.

[0021] At the beginning of each time slot, wireless charging is performed for each user;

[0022] in,

[0023] The process of transmitting wireless power to each user is as follows:

[0024] T k，0 +T k,1 ≤(1-α k )Γ

[0025]

[0026] in,

[0027] T k，0 This indicates the time each user spends training the target model locally each time;

[0028] T k,1 This indicates the time each user takes to transmit the target model each time;

[0029] This indicates the wireless charging time for each user;

[0030] α k ∈[0,1] represents the proportion of charging time for each user;

[0031] Γ represents the duration of one round of federated learning;

[0032] in,

[0033]

[0034]

[0035] in,

[0036] D k This indicates the dataset size for each user;

[0037] c represents the CPU cycles required to compute one sample of data;

[0038] This indicates the number of rounds the local user trains in a federated learning round;

[0039] f k This represents each user's computing power;

[0040] L k This represents the model size for each user;

[0041] R k This indicates the transmission rate for each user.

[0042] Preferably, updating the radio frequency power corresponding to the wireless bandwidth obtained by each user from the hybrid access point includes:

[0043] Sort the radio frequency power required by each user;

[0044] Based on the ranking of the radio frequency power required by each user, update the radio frequency power corresponding to the wireless bandwidth obtained by each user from the hybrid access point.

[0045] A wireless power transfer device, comprising:

[0046] The transmission unit is used to continuously transmit wireless energy to each user so that each user can train the target model locally and transmit the target model that has been trained locally.

[0047] The distribution unit is used to receive each target model uploaded by each user, aggregate each target model into a target global model, and then distribute it to each user.

[0048] The judgment unit is used to determine whether the instantaneous channel state information or statistical channel state information of the wireless link in the energy transmission system has been obtained;

[0049] The first optimization unit is used to optimize the wireless energy resource allocation strategy for each user by adopting a preset resource allocation strategy based on a deep Q-learning network and a greedy algorithm when the execution result of the judgment unit determines that the instantaneous channel state information of the wireless link in the energy transmission system has been obtained.

[0050] Preferably, the device further includes:

[0051] The second optimization unit is used to optimize the wireless energy resource allocation strategy for each user by adopting a resource allocation strategy based on a connected deep Q-learning network and the golden section method when the execution result of the judgment unit determines that only the statistical channel state information of the wireless link in the energy transmission system is known.

[0052] A wireless power transfer device includes: one or more processors, and a memory;

[0053] The memory stores computer-readable instructions that, when executed by the one or more processors, implement the steps of any of the wireless power transfer methods described above.

[0054] A readable storage medium storing computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of any of the wireless power transfer methods described above.

[0055] As can be seen from the technical solutions described above, when users distributed in a lightweight Internet of Things want to upload model data, the method provided in this application embodiment can continuously transmit wireless energy to each user so that each user can train the target model locally and transmit the locally trained target model. Furthermore, it can determine whether the instantaneous channel state information or statistical channel state information of the wireless link in the energy transmission system is known, so that the wireless energy resource allocation strategy for each user can be adjusted according to the instantaneous channel state information or statistical channel state information of the wireless link in the energy transmission system. If the instantaneous channel state information of the wireless link in the energy transmission system is known, a resource allocation strategy based on a deep Q-learning network and a greedy algorithm can be adopted to optimize the wireless energy resource allocation strategy for each user.

[0056] As described above, when users distributed across a lightweight Internet of Things (IoT) want to upload model data, the method provided in this application can effectively protect user data privacy while ensuring that each user can receive wireless energy from the hybrid access point of the energy transmission system to train and transmit the federated learning model. This effectively avoids the problem that users participating in federated learning are forced to interrupt federated learning due to excessive energy consumption or long execution delays, thus preventing them from uploading model parameters. Attached Figure Description

[0057] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0058] Figure 1 A schematic diagram of a federated learning framework for wireless power transfer assistance in digital power grids, provided for an embodiment of this application;

[0059] Figure 2 A flowchart illustrating a method for implementing wireless power transfer, as provided in this application embodiment;

[0060] Figure 3 The simulation results show how the test accuracy of the model trained using the federated learning framework provided in this application changes with the time span of each round of federated learning.

[0061] Figure 4 The simulation results show the variation of the test accuracy of the model trained using the federated learning framework provided in the embodiments of this application with the total system bandwidth.

[0062] Figure 5 This is a schematic diagram of a wireless power transmission device as an example of an embodiment of this application;

[0063] Figure 6 This is a hardware structure block diagram of a wireless power transmission device disclosed in an embodiment of this application. Detailed Implementation

[0064] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0065] Given that most current wireless power transfer solutions are ill-suited to complex and ever-changing business needs, this applicant has developed a wireless power transfer solution. When users distributed across a lightweight Internet of Things (IoT) want to upload model data, the method provided in this application can effectively protect user data privacy while ensuring that each user can receive wireless power from the hybrid access point of the power transfer system for training and transmitting federated learning models. This effectively avoids the problem that users participating in federated learning are forced to interrupt federated learning due to excessive energy consumption or long execution delays, thus preventing them from uploading model parameters.

[0066] The methods provided in this application can be used in a variety of general-purpose or special-purpose computing device environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor devices, distributed computing environments including any of the above devices, etc.

[0067] This application provides a wireless power transmission method, which can be applied to various energy management systems or digital power grid management systems, as well as to various computer terminals or smart terminals. The executing entity can be the processor or server of the computer terminal or smart terminal.

[0068] The following is combined Figure 1 and Figure 2 This application describes the flow of the wireless power transfer method according to its embodiments, such as... Figure 2 As shown, the process may include the following steps:

[0069] Step S101: Continuously transmit wireless energy to each user so that each user can train the target model locally and transmit the target model that has been trained locally.

[0070] Specifically, due to the development of artificial intelligence, centralized deep learning often requires collecting large amounts of data from distributed users and uploading them to a central server to support centralized training, which brings serious privacy risks. Therefore, more and more users are unwilling to share their private data, leading to a situation of data silos.

[0071] To address this issue, some researchers first proposed the concept of federated learning in 2016, aiming to promote decentralized intelligence without compromising privacy. In a federated learning framework, users contribute data by uploading model parameters, rather than their own raw private data, to collaborate with a federated learning server in training a global model. This ensures the security of private data while reducing communication overhead.

[0072] In recent years, with the development of federated learning, many studies on the challenges of applied federated learning have emerged. Among them, the energy consumption of users during the federated learning process is a key issue for wireless networks, including federated learning networks. It determines how long users in the network can work and whether they will be forced to exit federated learning due to insufficient energy. The number of users participating in federated learning is directly related to the performance of the system.

[0073] In practical applications, when users in a lightweight Internet of Things (IoT) participate in federated learning training, they may exit the federated learning process due to insufficient power supply or excessive execution latency.

[0074] For example,

[0075] Figure 1 A schematic diagram of a federated learning framework for wireless power transfer assistance in digital power grids, provided for an embodiment of this application;

[0076] Therefore, in order to solve this problem, the method provided in this application embodiment can continuously transmit wireless energy to each user so that each user can train the target model locally and transmit the locally trained target model.

[0077] in,

[0078] Each user can collect wireless energy through the hybrid access point of the wireless energy transmission system, and use the collected wireless energy to support each user in training a federated learning model locally in conjunction with the federated learning server, and then upload the trained federated learning model to the server.

[0079] For example,

[0080] In practical applications, time-division multiplexing can be used to continuously transmit wireless energy to each user, so that the wireless energy transmission system can ensure both the training and uploading of local models within a limited time, and also ensure that each user can collect wireless energy from the hybrid access point.

[0081] In practical applications, after utilizing the collected wireless energy, each user can use the collected wireless transmission energy to train the federated learning model and then send the trained federated learning model to the server.

[0082] Therefore, the method provided in this application embodiment can also receive various target models uploaded by various users, wherein,

[0083] The target model can be a federated learning model trained by the user using the collected wireless energy. The target model includes parameters corresponding to the user's original private data.

[0084] After collecting the target models, in order to protect user privacy, the collected target models can be further aggregated into a global model, and the aggregated global model can be distributed to each user for further learning and training, so as to achieve secure and reliable data upload to the server for users.

[0085] Step S102: Determine whether the instantaneous channel state information or statistical channel state information of the wireless link in the energy transmission system has been obtained.

[0086] Specifically, as can be seen from the above description, the method provided in this application embodiment can continuously transmit wireless energy to each user.

[0087] In practical applications, the time, power, and bandwidth resources available to the energy transmission system are limited. Therefore, to ensure that as many users as possible can meet the resource needs of those participating in federated learning, and to ensure that more users can continue to participate in federated learning with limited resources, it is necessary to allocate resources to each user based on the limited resources available.

[0088] Therefore, different resource allocation schemes need to be implemented based on the available resources and the needs of each user.

[0089] In practical applications, the channel state information of the wireless link in the energy transmission system can affect the resource allocation to each user.

[0090] Therefore, it can be determined whether the instantaneous channel state information or statistical channel state information of the wireless link in the energy transmission system is known, so as to determine which resource allocation strategy can be used to allocate resources to each user in order to maximize resource utilization.

[0091] in,

[0092] The channel state information of the wireless link in the energy transfer system can include instantaneous channel state information and statistical channel state information.

[0093] Different types of channel state information employ different resource allocation strategies.

[0094] in,

[0095] The statistical channel state information and instantaneous channel state information of the wireless link in the wireless power transmission system can be obtained by the receiver.

[0096] Instantaneous channel state information of a wireless link can reflect the quality of the wireless link at a specific moment, while statistical channel state information can reflect the distribution and statistical characteristics of the wireless link.

[0097] When each user terminal sends uplink pilot signals to the receiver, the receiver can perform channel estimation to obtain the uplink and downlink channel information of the system. From this, the instantaneous channel state information of the wireless link of the energy transmission system can be determined.

[0098] In practical applications, when the wireless link changes too rapidly, it may not be possible to obtain the instantaneous channel state information of the wireless link through channel estimation in a short period of time. In such cases, statistical characteristics can be used to obtain statistical channel state information.

[0099] Therefore, if the instantaneous channel state information of the wireless link in the energy transmission system is known, step S103 can be executed.

[0100] If only the statistical channel status information of the wireless link in the energy transfer system is known, then step S104 can be executed.

[0101] Step S103: Optimize the wireless energy resource allocation strategy for each user by adopting a resource allocation strategy based on a deep Q-learning network and a greedy algorithm.

[0102] Specifically, as can be seen from the above description, the method provided in this application embodiment can adopt corresponding resource allocation strategies by analyzing the channel state information of the wireless link of the energy transmission system.

[0103] If the instantaneous channel state information of the wireless link in the energy transmission system is known, it means that the wireless link quality at a specific moment can be understood through the instantaneous channel state information of the wireless link. Therefore, the resource allocation strategy for each user can be analyzed through the instantaneous channel state information of the wireless link in the energy transmission system.

[0104] Therefore, when the instantaneous channel state information of the wireless link is obtained, a resource allocation strategy based on deep Q-learning network and greedy algorithm can be adopted to optimize the wireless energy resource allocation strategy for each user.

[0105] in,

[0106] By employing a resource allocation strategy based on deep Q-learning networks and greedy algorithms, individual users in lightweight IoT systems can meet the requirements of energy consumption and latency to participate in federated training.

[0107] in,

[0108] The wireless power transfer system can determine the energy consumption and latency thresholds for each user in a lightweight Internet of Things (IoT) based on actual network usage, thereby determining whether each user in the lightweight IoT can meet the energy consumption and latency requirements.

[0109] Step S104: Optimize the wireless energy resource allocation strategy for each user by adopting a resource allocation strategy based on a connected deep Q-learning network and the golden section method.

[0110] Specifically, as can be seen from the above description, the method provided in this application embodiment can adopt different resource allocation strategies by analyzing the channel state information of the wireless link of the energy transmission system.

[0111] In practical applications, when the wireless link changes too rapidly, it may not be possible to obtain the instantaneous channel state information of the wireless link through channel estimation in a short period of time. In such cases, statistical characteristics can be used to obtain statistical channel state information.

[0112] If the instantaneous channel state information of the wireless link in the energy transmission system is known, it means that the instantaneous channel state information of the wireless link may not be available in time through channel estimation. However, the resource allocation strategy for each user can be analyzed by using the statistical channel state information of the wireless link.

[0113] Therefore, when the statistical channel state information of the wireless link is obtained, a resource allocation strategy based on the connected deep Q-learning network and the golden section method can be adopted to optimize the wireless energy resource allocation strategy for each user.

[0114] in,

[0115] By using a resource allocation strategy based on a connected deep Q-learning network and the golden ratio method, individual users in a lightweight Internet of Things can meet the requirements of energy consumption and latency to participate in federated training.

[0116] in,

[0117] The wireless power transfer system can determine the energy consumption and latency thresholds for each user in a lightweight Internet of Things (IoT) based on actual network usage, thereby determining whether each user in the lightweight IoT can meet the energy consumption and latency requirements.

[0118] As can be seen from the above-described technical solutions, when users distributed in a lightweight Internet of Things want to upload model data, the method provided in this application embodiment can effectively protect the user's data privacy while ensuring that each user can receive wireless energy from the hybrid access point of the energy transmission system to train and transmit the federated learning model. This effectively avoids the problem that users participating in federated learning are forced to interrupt federated learning due to excessive energy consumption or excessive execution latency, and are unable to upload model parameters.

[0119] As can be seen from the technical solutions described above, the method provided in this application embodiment can optimize the wireless energy resource allocation strategy for each user by utilizing a resource allocation strategy based on a deep Q-learning network and a greedy algorithm. The following describes the creation process of the resource allocation strategy based on a deep Q-learning network and a greedy algorithm, which may include the following steps:

[0120] Step S201: Dynamically update the wireless bandwidth obtained by each user from the hybrid access point of the energy transmission system.

[0121] Specifically, in practical applications, the energy collected by users from the hybrid access point of the energy transmission system is affected by the wireless bandwidth obtained by each user at the hybrid access point.

[0122] In actual applications, the wireless bandwidth obtained by each user at the hybrid access point is different.

[0123] Therefore, in practical applications, the wireless transmission system can acquire instantaneous channel state information of the wireless link in each federated training round. After acquiring the instantaneous channel state information of the wireless link, the deep Q-learning network can be used to dynamically update the wireless bandwidth obtained by each user from the hybrid access point of the energy transmission system, so as to adjust the resource allocation strategy for each user in real time according to the wireless bandwidth resources obtained by each user.

[0124] Step S202: Update the radio frequency power corresponding to the wireless bandwidth obtained by each user from the hybrid access point.

[0125] Specifically, as described above, the method provided in this application embodiment utilizes a deep Q-learning network to dynamically update the wireless bandwidth obtained by each user from the hybrid access point of the energy transmission system.

[0126] After updating the wireless bandwidth obtained by each user from the hybrid access point, the corresponding radio frequency power of the wireless bandwidth obtained by each user from the hybrid access point can be further updated so as to determine the radio frequency power required by each user. This allows for timely adjustment of the resource allocation strategy for each user based on the wireless bandwidth resources obtained by each user and the radio frequency power required by each user.

[0127] in,

[0128] In practical applications, the radio frequency power required by each user can be sorted according to the actual application needs, and then the radio frequency power corresponding to the wireless bandwidth obtained by each user from the hybrid access point can be updated according to the sorting results of the radio frequency power required by each user.

[0129] For example,

[0130] Users can be sorted by their required RF power from smallest to largest, and then the required RF power for each user can be updated sequentially.

[0131] Step S203: Based on the updated wireless bandwidth and RF power obtained by each user from the hybrid access point, update the wireless charging time of each user at the hybrid access point until the loss function of the preset first resource allocation network model converges, and obtain the resource allocation strategy of the energy transmission system based on deep Q-learning network and greedy algorithm.

[0132] Specifically, as described above, the method provided in this application embodiment can dynamically update the wireless bandwidth and required radio frequency power obtained by each user from the hybrid access point using a deep Q-learning network.

[0133] Once the wireless bandwidth and required RF power for each user from the hybrid access point are determined, the wireless charging time for each user at the hybrid access point can be further determined.

[0134] Therefore, after updating the wireless bandwidth and corresponding RF power obtained by each user at the hybrid access point, the wireless charging time of each user at the hybrid access point can be dynamically updated using a deep Q-learning network until the loss function of the first resource allocation network model converges. This allows us to obtain the resource allocation strategy of the energy transmission system based on a deep Q-learning network and a greedy algorithm.

[0135] in,

[0136] The first resource allocation network model can be trained using the wireless bandwidth and corresponding RF power obtained by each user from the hybrid access point and the wireless charging time of each user at the hybrid access point as training samples, and the resource allocation strategy corresponding to the wireless bandwidth and corresponding RF power obtained by each user from the hybrid access point and the wireless charging time of each user at the hybrid access point as sample labels.

[0137] For example,

[0138] In practical applications, a sufficient number of training rounds can be set to ensure that the loss function of the first resource allocation model can converge.

[0139] In practical applications, once the loss function of the first resource allocation model converges, a more stable resource allocation strategy can be obtained.

[0140] For example, the wireless charging time for each user, as well as the specific wireless bandwidth and RF power obtained by each user from the hybrid access point, can be obtained through the resource allocation strategy.

[0141] For example,

[0142] In each federated training round, the system can acquire instantaneous channel state information of the wireless link and dynamically update the wireless bandwidth obtained by each user from the hybrid access point using a deep Q-learning network.

[0143] First, Markov modeling is performed, and the state space of the deep Q-learning network is defined as follows:

[0144] S(γ e )={B1(γ e ),...,B k (γ e ),...,B K (γ e )} (1)

[0145] in,

[0146] γ e This can be represented as the training epoch number of a deep Q-learning network;

[0147] B k (γ e ) can represent user k in γ e The result of round-robin bandwidth allocation;

[0148] The method provided in this application embodiment can define the action space of a deep Q-learning network as follows:

[0149] A(γ e )={a i (γ - {e})|i∈[1,2K]}, (2)

[0150] in,

[0151]

[0152] i can represent the sequence number of the action command;

[0153] K can represent the number of users participating in edge federated learning;

[0154] ξ can represent positive feedback;

[0155] -ξ can represent negative feedback;

[0156] In practical applications, after selecting a certain action, each user's wireless bandwidth will be updated accordingly, as follows:

[0157] B k (γ e +1)=B k (γ e )+a i (γ e (3)

[0158] in,

[0159] B k (γ e ) can represent user k in γ e The result of round-robin bandwidth allocation;

[0160] a i (γ e ) can be represented in γ e The updated value of bandwidth allocation under the action of wheel i;

[0161] After each user's wireless bandwidth is updated, a greedy algorithm can be used to update the corresponding RF power based on each user's latency and power consumption limits, as follows:

[0162]

[0163] in,

[0164]

[0165] and,

[0166]

[0167] in,

[0168] P k，2 This can be expressed as the radio frequency transmit power of a hybrid access point;

[0169] This can be expressed as the updated RF transmit power of the hybrid access point;

[0170] P k,0 It can represent the user's local training power;

[0171] D k It can represent the dataset size for each user;

[0172] c can represent the CPU cycles required to compute one sample of data;

[0173] It can represent the number of rounds the local user trains in a federated learning round;

[0174] f k It can represent each user's computing power;

[0175] L k It can represent the model size for each user;

[0176] B k This can represent the wireless bandwidth allocated to each user from the hybrid access point;

[0177] P k,1 This can be expressed as the transmit power for each user;

[0178] hk can be represented as the channel parameters of the wireless link between the k-th user and the hybrid access point;

[0179] σ 2 This can be expressed as the variance of additive white Gaussian noise;

[0180] η∈[0,1] can represent the user's energy harvesting efficiency;

[0181] θ2 can represent the lower limit of the unloading coefficient;

[0182] θ1 can represent the upper limit of the unloading coefficient;

[0183] Γ can represent the duration of a round of federated learning;

[0184] This application embodiment first sorts users according to their required RF power from smallest to largest, and then updates the RF power for each user in sequence, as detailed below:

[0185]

[0186] After each user's RF power is updated, the method provided in this application embodiment can further update the corresponding wireless charging time, specifically as follows:

[0187] a k =θ1 (8)

[0188] After the user's wireless bandwidth, RF power, and charging time are updated, the number of user participants in each federated learning round can be obtained. The deep Q-learning network can then be defined as the difference in the number of user participants between two adjacent federated learning rounds, and its loss function can be defined as follows:

[0189] V(γ e )=(Y(γ e )-Q(S(γ e ),A(γ e );ω dqn (γe ))) 2 (9)

[0190] in,

[0191] Q(S(γ e ),A(γ e );ω dqn (γ e )) can represent a state action value function;

[0192] Y(γ e () can represent the objective function;

[0193] in,

[0194]

[0195] in,

[0196] δ can represent the discount factor;

[0197] It can represent the γth e The model parameters of the target network in the next round;

[0198] In practical applications, the above process can be run until the loss function of the deep Q-learning network converges, and the resource allocation strategy of the wireless power transmission system based on the instantaneous channel information of the wireless link can be obtained.

[0199] As can be seen from the above-described technical solutions, once the instantaneous channel state information of the wireless link of the wireless transmission energy system is determined, the method provided in this application embodiment can utilize a resource allocation strategy based on a deep Q-learning network and a greedy algorithm to optimize the wireless energy resource allocation strategy for each user, so as to better maximize resource utilization and ensure that as many users as possible can participate in federated learning.

[0200] As can be seen from the technical solutions described above, the method provided in this application embodiment can optimize the wireless energy resource allocation strategy for each user by utilizing a resource allocation strategy based on a connected deep Q-learning network and the golden section method. The following describes the creation process of the resource allocation strategy based on a connected deep Q-learning network and the golden section method, which may include the following steps:

[0201] Step S301: Dynamically update the wireless bandwidth obtained by each user from the hybrid access point of the energy transmission system.

[0202] Specifically, in practical applications, the energy collected by users from the hybrid access point of the energy transmission system is affected by the wireless bandwidth obtained by each user at the hybrid access point.

[0203] In actual applications, the wireless bandwidth obtained by each user at the hybrid access point is different.

[0204] Therefore, in practical applications, the wireless transmission system can obtain the statistical channel state information of the wireless link in each federated training round. After obtaining the statistical channel state information of the wireless link, the deep Q-learning network can be used to dynamically update the wireless bandwidth obtained by each user from the hybrid access point of the energy transmission system, so as to adjust the resource allocation strategy for each user in real time according to the wireless bandwidth resources obtained by each user.

[0205] Step S302: Update the radio frequency power corresponding to the wireless bandwidth obtained by each user from the hybrid access point.

[0206] Specifically, as described above, the method provided in this application embodiment utilizes a deep Q-learning network to dynamically update the wireless bandwidth obtained by each user from the hybrid access point of the energy transmission system.

[0207] After updating the wireless bandwidth obtained by each user from the hybrid access point, the corresponding radio frequency power of the wireless bandwidth obtained by each user from the hybrid access point can be further updated so as to determine the radio frequency power required by each user. This allows for timely adjustment of the resource allocation strategy for each user based on the wireless bandwidth resources obtained by each user and the radio frequency power required by each user.

[0208] in,

[0209] In practical applications, the radio frequency power required by each user can be sorted according to the actual application needs, and then the radio frequency power corresponding to the wireless bandwidth obtained by each user from the hybrid access point can be updated according to the sorting results of the radio frequency power required by each user.

[0210] For example,

[0211] Users can be sorted by their required RF power from smallest to largest, and then the required RF power for each user can be updated sequentially.

[0212] Step S303: Based on the updated wireless bandwidth and corresponding RF power obtained by each user from the hybrid access point, update the corresponding wireless charging time of each user at the hybrid access point until the loss function of the preset second resource allocation model network converges, and obtain the resource allocation strategy of the energy transmission system based on the connected deep Q-learning network and the golden section method.

[0213] Specifically, as described above, the method provided in this application embodiment can dynamically update the wireless bandwidth and required radio frequency power obtained by each user from the hybrid access point using a deep Q-learning network.

[0214] Once the wireless bandwidth and required RF power for each user from the hybrid access point are determined, the wireless charging time for each user at the hybrid access point can be further determined.

[0215] Therefore, after updating the wireless bandwidth and corresponding RF power obtained by each user at the hybrid access point, the wireless charging time of each user at the hybrid access point can be dynamically updated using a deep Q-learning network until the loss function of the second resource allocation network model converges. This yields the resource allocation strategy of the energy transmission system based on the deep Q-learning network and the golden section method.

[0216] in,

[0217] The second resource allocation network model can be trained using the wireless bandwidth and corresponding RF power obtained by each user from the hybrid access point and the wireless charging time of each user at the hybrid access point as training samples, and the resource allocation strategy corresponding to the wireless bandwidth and corresponding RF power obtained by each user from the hybrid access point and the wireless charging time of each user at the hybrid access point as sample labels.

[0218] For example,

[0219] In practical applications, a sufficient number of training rounds can be set to ensure that the loss function of the second resource allocation model can converge.

[0220] In practical applications, once the loss function of the second resource allocation model converges, a relatively stable resource allocation strategy can be obtained. For example, the wireless charging time for each user, as well as the specific amount of wireless bandwidth and RF power obtained by each user from the hybrid access point, can be obtained through the obtained resource allocation strategy.

[0221] For example,

[0222] In each federated training round, the system can acquire statistical channel state information of the wireless link and dynamically update the wireless bandwidth and RF power obtained by each user from the hybrid access point using a deep Q-learning network.

[0223] in,

[0224] We can first perform Markov modeling, defining the state space of the deep Q-learning network as follows:

[0225] S(γe )={B1(γ e ),...,B k (γ e ),...,B T (γ e ), P 1，2 (γ e ),...,P k，2 (γ e ),...,P T，2 (γ e (11)

[0226] The action space of a deep Q-learning network can be defined as follows:

[0227] A(γ e )={a j (γ - {e})|j∈[1,4K]} (12)

[0228] in,

[0229] j can represent the sequence number of the action command, and the definition of each action command is the same as described above;

[0230] In practical applications, after a certain action is selected, each user's wireless bandwidth and RF power are updated accordingly, as follows:

[0231] B k (γ e +1)=B k (γ e )+a j (γ e (13)

[0232] P k，2 (γ e +1)=P k，2 (γ e )+a j (γ e (14)

[0233] After the above two aspects are updated, the method provided in this application embodiment can transform the problem of wireless charging time allocation into the following:

[0234]

[0235] in,

[0236]

[0237] θ3 can represent an intermediate variable;

[0238] in,

[0239] This can be represented as the channel state threshold obtained by each user based on energy consumption constraints;

[0240] Furthermore, after updating the wireless bandwidth and RF power of each user, the method provided in this application embodiment can further utilize the golden ratio method to update the wireless charging time of each user.

[0241] After updating the wireless bandwidth, charging power, and charging time for each user, the statistical number of users participating in federated learning can be obtained. The reward for the deep Q-learning network is then defined as the difference in the statistical number of participating users between two adjacent federated learning rounds. By running this process until the loss function of the deep Q-learning network converges, the system's resource allocation strategy based on statistical channel information can be obtained.

[0242] For example,

[0243] Figure 3 The simulation results illustrate how the test accuracy of a model trained using the federated learning framework provided in this application varies with the time span of each round of federated learning.

[0244] Figure 4 The simulation results of the test accuracy of the model trained using the federated learning framework provided in the embodiments of this application as a function of the total system bandwidth are illustrated.

[0245] Depend on Figure 3 and Figure 4 It can be seen that in the Python simulation environment, in the simulation experiment, parameters B = 100MHz and P = 20W, The variation range is [5, 25]s. By comparing the model trained by the federated learning framework based on the traditional resource allocation method, the model trained by the federated learning framework based on the resource allocation mechanism proposed by the method provided in this application has higher test accuracy. This phenomenon verifies the effectiveness of the method provided in this application.

[0246] In the simulation experiment, parameters P = 20W, and B varies in the range of [50, 150]s.

[0247] By comparing the models trained by the federated learning framework based on traditional resource allocation methods, the models trained by the federated learning framework based on the resource allocation mechanism proposed by the method provided in this application have higher test accuracy. This phenomenon verifies the effectiveness of the method provided in this application.

[0248] As can be seen from the above-described technical solutions, once the statistical channel state information of the wireless link of the wireless transmission energy system is determined, the method provided in this application embodiment can utilize a resource allocation strategy based on a federated deep Q-learning network and the golden section method to optimize the wireless energy resource allocation strategy for each user, so as to better maximize resource utilization and ensure that as many users as possible can participate in federated learning.

[0249] As can be seen from the technical solutions described above, the method provided in this application embodiment can continuously transmit wireless power to each user. The process will be described below, and it may include the following steps:

[0250] Step S401: Divide the transmission time for transmitting wireless energy to each user into several time slots of equal span.

[0251] Specifically, in practical applications, continuously charging each user may cause energy loss and equipment damage. Therefore, it is advisable to consider continuously charging different users according to their needs in order to maximize resource utilization.

[0252] For example,

[0253] The transmission time for transmitting wireless power to each user can be divided into several time slots of equal span, and time-division multiplexing can be used to transmit wireless power to each user.

[0254] Step S402: At the beginning of each time slot, wireless charging is performed on each user.

[0255] Specifically, as can be seen from the above description, the method provided in this application embodiment can divide the transmission time for transmitting wireless energy to each user into several time slots of equal span, and can use time division multiplexing to transmit wireless energy to each user.

[0256] Furthermore, after dividing the time for transmitting wireless energy into multiple time slots of equal span, wireless charging can be performed on each user at the beginning of each time slot.

[0257] in,

[0258] The process of transmitting wireless power to each user can be represented as follows:

[0259] T k，0 +T k,1 ≤(1-α k )Γ (17)

[0260]

[0261] in,

[0262] T k，0 It can represent the time each user spends training the target model locally each time;

[0263] T k,1 This can represent the time it takes for each user to transmit the target model each time;

[0264] S(γ e )={B1(γ e ),...,B k (γ e ),...,B T (γ e )} can represent the wireless charging time for each user;

[0265] α k ∈[0,1] can represent the proportion of charging time for each user;

[0266] Γ can represent the duration of a round of federated learning;

[0267] in,

[0268]

[0269]

[0270] in,

[0271] D k It can represent the dataset size for each user;

[0272] c can represent the CPU cycles required to compute one sample of data;

[0273] It can represent the number of rounds the local user trains in a federated learning round;

[0274] f k It can represent each user's computing power;

[0275] L k It can represent the model size for each user;

[0276] R k It can represent the transmission rate for each user;

[0277] also,

[0278]

[0279] B k This can represent the wireless bandwidth allocated to each user from the hybrid access point;

[0280] Pk,1 This can be expressed as the transmit power for each user;

[0281] hk can be represented as the channel parameters of the wireless link between the k-th user and the hybrid access point;

[0282] σ 2 This can be expressed as the variance of additive white Gaussian noise;

[0283] in,

[0284] The wireless energy collected by each user from the hybrid access point can be expressed as:

[0285]

[0286] η∈[0,1] can represent the user's energy harvesting efficiency;

[0287] P k,2 This can be expressed as the radio frequency transmit power of a hybrid access point;

[0288] in,

[0289] The training energy consumption and upload energy consumption for each user can be represented as follows:

[0290] E k,0 =P k,0 T k,0 (twenty three)

[0291] E k,1 =P k,1 T k,1 (twenty four)

[0292] E k,0 This can be expressed as the training energy consumption per user;

[0293] E k,1 This can be expressed as the upload energy consumption per user;

[0294] P k,0 This can be represented as the user's local training power.

[0295] As can be seen from the technical solutions described above, the method provided in this application embodiment can use time-division multiplexing to continuously transmit wireless energy to each user, so as to better maximize resource utilization and ensure that as many users as possible can participate in federated learning.

[0296] The wireless power transmission device provided in the embodiments of this application is described below. The wireless power transmission device described below can be referred to in correspondence with the wireless power transmission method described above.

[0297] See Figure 5 , Figure 5 This is a schematic diagram of the structure of a wireless power transmission device disclosed in an embodiment of this application.

[0298] like Figure 5 As shown, the wireless power transfer device may include:

[0299] The transmission unit 101 is used to continuously transmit wireless energy to each user so that each user can train the target model locally and transmit the target model that has been trained locally.

[0300] The judgment unit 102 is used to determine whether the instantaneous channel state information or statistical channel state information of the wireless link in the energy transmission system has been obtained;

[0301] The first optimization unit 103 is used to optimize the wireless energy resource allocation strategy for each user by adopting a preset resource allocation strategy based on a deep Q-learning network and a greedy algorithm when the execution result of the judgment unit determines that the instantaneous channel state information of the wireless link in the energy transmission system has been obtained.

[0302] As can be seen from the technical solutions described above, when users distributed in a lightweight Internet of Things want to upload model data, the device provided in this application embodiment can continuously transmit wireless energy to each user so that each user can train the target model locally and transmit the locally trained target model. Furthermore, it can determine whether the instantaneous channel state information or statistical channel state information of the wireless link in the energy transmission system is known, so that the wireless energy resource allocation strategy for each user can be adjusted according to the instantaneous channel state information or statistical channel state information of the wireless link in the energy transmission system. If the instantaneous channel state information of the wireless link in the energy transmission system is known, a preset resource allocation strategy based on a deep Q-learning network and a greedy algorithm can be adopted to optimize the wireless energy resource allocation strategy for each user.

[0303] As described above, when users distributed in a lightweight Internet of Things want to upload model data, the device provided in this application embodiment can effectively protect the user's data privacy while ensuring that each user can receive wireless energy from the hybrid access point of the energy transmission system to train and transmit the federated learning model. This effectively avoids the problem that users participating in federated learning are forced to interrupt federated learning due to excessive energy consumption or excessive execution latency, and are unable to upload model parameters.

[0304] Further optionally, the device may also include:

[0305] The second optimization unit is used to optimize the wireless energy resource allocation strategy for each user by adopting a resource allocation strategy based on a connected deep Q-learning network and the golden section method when the execution result of the judgment unit determines that only the statistical channel state information of the wireless link in the energy transmission system is known.

[0306] The specific processing flow of each unit included in the aforementioned wireless power transmission device can be found in the relevant introduction of the wireless power transmission method section above, and will not be repeated here.

[0307] The wireless power transfer device provided in this application embodiment can be applied to wireless power transfer devices, such as terminals: mobile phones, computers, etc. Optionally, Figure 6 A hardware block diagram of a wireless power transfer device is shown, with reference to... Figure 6 The hardware structure of a wireless power transmission device may include: at least one processor 1, at least one communication interface 2, at least one memory 3, and at least one communication bus 4.

[0308] In this embodiment, the number of processor 1, communication interface 2, memory 3, and communication bus 4 is at least one, and processor 1, communication interface 2, and memory 3 communicate with each other through communication bus 4.

[0309] Processor 1 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application.

[0310] Memory 3 may include high-speed RAM, and may also include non-volatile memory, such as at least one disk storage device;

[0311] The memory stores a program, which the processor can call. The program is used to implement the various processing steps in the aforementioned terminal wireless power transmission scheme.

[0312] This application embodiment also provides a readable storage medium that can store a program suitable for execution by a processor, the program being used to implement various processing flows of the aforementioned terminal in a wireless power transmission scheme.

[0313] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0314] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0315] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Various embodiments can be combined with each other. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A wireless power transfer method, characterized in that, include: Continuously transmit wireless power to each user so that each user can train the target model locally and transmit the locally trained target model. Determine whether instantaneous channel state information or statistical channel state information of the wireless link in the energy transmission system has been obtained; If the instantaneous channel state information of the wireless link in the energy transmission system is known, a resource allocation strategy based on a deep Q-learning network and a greedy algorithm is adopted to optimize the wireless energy resource allocation strategy for each user. If only the statistical channel state information of the wireless link in the energy transmission system is known, a resource allocation strategy based on deep Q-learning network and golden section method is adopted to optimize the wireless energy resource allocation strategy for each user. The process of creating the resource allocation strategy based on deep Q-learning networks and greedy algorithms includes: Dynamically update the wireless bandwidth obtained by each user from the hybrid access point of the energy transmission system; Update the radio frequency power corresponding to the wireless bandwidth obtained by each user from the hybrid access point; Based on the updated wireless bandwidth and RF power obtained by each user from the hybrid access point, the wireless charging time of each user at the hybrid access point is updated until the loss function of the preset first resource allocation network model converges, thereby obtaining the resource allocation strategy of the energy transmission system based on a deep Q-learning network and a greedy algorithm. The preset first resource allocation network model is trained using the wireless bandwidth and corresponding RF power obtained by each user from the hybrid access point and the wireless charging time of each user at the hybrid access point as training samples, and the resource allocation strategy corresponding to the wireless bandwidth and corresponding RF power obtained by each user from the hybrid access point and the wireless charging time of each user at the hybrid access point as sample labels.

2. The method according to claim 1, characterized in that, The process of creating the resource allocation strategy based on deep Q-learning networks and the golden section method includes: Dynamically update the wireless bandwidth obtained by each user from the hybrid access point of the energy transmission system; Update the radio frequency power corresponding to the wireless bandwidth obtained by each user from the hybrid access point; Based on the updated wireless bandwidth and corresponding RF power obtained by each user from the hybrid access point, the wireless charging time of each user at the hybrid access point is updated until the loss function of the preset second resource allocation model network converges, thereby obtaining the resource allocation strategy of the energy transmission system based on the deep Q-learning network and the golden section method. The preset second resource allocation network model is trained using the wireless bandwidth and corresponding RF power obtained by each user from the hybrid access point and the wireless charging time of each user at the hybrid access point as training samples, and the resource allocation strategy corresponding to the wireless bandwidth and corresponding RF power obtained by each user from the hybrid access point and the wireless charging time of each user at the hybrid access point as sample labels.

3. The method according to claim 1, characterized in that, The continuous transmission of wireless power to each user includes: The transmission time for wireless power to each user is divided into several time slots of equal span. At the beginning of each time slot, wireless charging is performed for each user; in, The process of transmitting wireless power to each user is as follows: in, This indicates the time each user spends training the target model locally each time; This indicates the time each user takes to transmit the target model each time; This indicates the wireless charging time for each user; This represents the percentage of charging time for each user. Indicates the duration of one round of federated learning; in, in, This indicates the dataset size for each user; This represents the CPU cycles required to compute one sample of data. This indicates the number of rounds the local user trains in a federated learning round; This represents each user's computing power; This represents the model size for each user; This indicates the transmission rate for each user.

4. The method according to claim 1 or 2, characterized in that, The update of the radio frequency power corresponding to the wireless bandwidth obtained by each user from the hybrid access point includes: Sort the radio frequency power required by each user; Based on the ranking of the radio frequency power required by each user, update the radio frequency power corresponding to the wireless bandwidth obtained by each user from the hybrid access point.

5. A wireless power transmission device, characterized in that, include: The transmission unit is used to continuously transmit wireless energy to each user so that each user can train the target model locally and transmit the target model that has been trained locally. The judgment unit is used to determine whether the instantaneous channel state information or statistical channel state information of the wireless link in the energy transmission system has been obtained; The first optimization unit is used to optimize the wireless energy resource allocation strategy for each user by adopting a preset resource allocation strategy based on a deep Q-learning network and a greedy algorithm when the execution result of the judgment unit is that the instantaneous channel state information of the wireless link in the energy transmission system has been obtained. The second optimization unit is used to optimize the wireless energy resource allocation strategy for each user by adopting a resource allocation strategy based on deep Q-learning network and golden section method when the execution result of the judgment unit determines that only the statistical channel state information of the wireless link in the energy transmission system is known. The process of creating the resource allocation strategy based on deep Q-learning networks and greedy algorithms includes: Dynamically update the wireless bandwidth obtained by each user from the hybrid access point of the energy transmission system; Update the radio frequency power corresponding to the wireless bandwidth obtained by each user from the hybrid access point; Based on the updated wireless bandwidth and RF power obtained by each user from the hybrid access point, the wireless charging time of each user at the hybrid access point is updated until the loss function of the preset first resource allocation network model converges, thereby obtaining the resource allocation strategy of the energy transmission system based on a deep Q-learning network and a greedy algorithm. The preset first resource allocation network model is trained using the wireless bandwidth and corresponding RF power obtained by each user from the hybrid access point and the wireless charging time of each user at the hybrid access point as training samples, and the resource allocation strategy corresponding to the wireless bandwidth and corresponding RF power obtained by each user from the hybrid access point and the wireless charging time of each user at the hybrid access point as sample labels.

6. A wireless power transmission device, characterized in that, include: One or more processors, and memory; The memory stores computer-readable instructions that, when executed by the one or more processors, implement the steps of the wireless power transfer method as described in any one of claims 1 to 4.

7. A readable storage medium, characterized in that: The readable storage medium stores computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of the wireless power transfer method as described in any one of claims 1 to 4.

Images