Underground wireless energy supply sensor network system, signal transmission method, device and medium thereof

Abstract

The present application relates to a kind of underground wireless energy supply sensor network systems and its signal transmission method, equipment and medium, comprising: the deployment position of reconfigurable intelligent hypersurface is optimized offline;Underground wireless energy supply sensor network current network state parameters are obtained in optimal deployment position, including each underground sensor node downlink state, uplink state, reconfigurable intelligent hypersurface auxiliary link state and underground sensor node current energy information collected;Pre-trained depth reinforcement learning model suitable for continuous action space is used to jointly optimize reconfigurable intelligent hypersurface phase configuration, downlink time slot allocation and uplink transmission power allocation, and the optimal joint control strategy of current transmission frame is obtained and executed;After current transmission frame is executed, update network state parameters, and carry out optimization transmission control of next transmission frame.Compared with prior art, the present application considers link reliability, overall network throughput and system long-term sustainable operation ability.

Description

Technical Field

[0001] This invention relates to the field of underground wireless power supply sensor networks, and in particular to an underground wireless power supply sensor network system and its signal transmission method, equipment and medium. Background Technology

[0002] Underground wireless sensor networks (WSNs) are a type of wireless sensor network that buries sensor nodes in soil or underground media. They can be applied to precision agriculture, underground infrastructure monitoring, environmental sensing, and smart cities. Compared to traditional above-ground WSNs, the electromagnetic wave propagation environment in underground WSNs is more complex. Signals are affected by soil attenuation, multipath effects, changes in dielectric parameters, and burial depth when propagating in the soil. Therefore, these systems generally suffer from problems such as high link loss, insufficient communication reliability, and limited node energy.

[0003] The underground wireless power supply sensor network consists of a surface-mounted radio frequency (RF) power source, a receiver, and multiple underground sensor nodes. The underground sensor nodes acquire energy from the RF power source in the downlink using RF energy harvesting technology, and then use the harvested energy to complete the subsequent uplink transmission of sensor information. This process typically employs a "harvest energy first, then transmit" approach. This method alleviates the limited battery life of underground sensor nodes to some extent, facilitating long-term deployment and continuous monitoring.

[0004] However, existing underground wireless power supply sensor networks still have significant shortcomings. Because underground sensor nodes are buried in the soil, which strongly attenuates electromagnetic waves, the downlink energy reaching and being effectively utilized is very limited. This often results in nodes requiring extended periods of energy harvesting to support reliable subsequent data transmission. As communication distance or burial depth increases, the constraint between node energy acquisition and data upload capabilities intensifies, easily leading to a decrease in overall system throughput and increased communication latency. Furthermore, because the "harvest-then-transmit" approach requires nodes to accumulate energy before initiating active communication, traditional underground wireless power supply sensor networks are poorly adapted to scenarios involving sudden data uploads.

[0005] A search revealed Chinese invention patent application CN113660628A, which discloses a method for optimizing the throughput of underground wireless power supply sensor networks. This method combines backscatter communication with radio frequency energy harvesting to form a backscatter-assisted underground wireless power supply sensor network. Compared with the simple method of harvesting energy first and then transmitting, the backscatter-assisted scheme can transmit information simultaneously during wireless power supply, which helps improve system time utilization and, to some extent, meets the demand for bursty information transmission. However, the performance of the backscatter-assisted scheme in underground environments is still limited: on the one hand, the radio frequency signal incident on underground nodes is significantly weakened after passing through the soil, resulting in insufficient energy base available for backscattering and active transmission. On the other hand, practical energy harvesting circuits usually have nonlinear characteristics, resulting in low energy conversion efficiency under low input power conditions, which further compresses the available energy of the nodes. In addition, changes in soil moisture, temperature, and dielectric parameters cause fluctuations in propagation conditions, affecting both downlink wireless power supply and uplink data transmission. It is clear that relying solely on the combination of backscattering and energy harvesting is still insufficient to fundamentally overcome the problems caused by high attenuation and time-varying channels underground.

[0006] Therefore, there is an urgent need to design a new underground wireless power supply sensor network scheme to address the problems of large underground link attenuation, nonlinear energy harvesting process, strong coupling between downlink power supply and uplink transmission, and insufficient adaptability of traditional static optimization methods in dynamic soil environments. Summary of the Invention

[0007] The purpose of this invention is to overcome the shortcomings of the existing technology and provide an underground wireless power supply sensor network system and its signal transmission method, device and medium. By introducing a reconfigurable smart metasurface, the phase configuration, downlink time slot allocation and uplink transmit power allocation of the reconfigurable smart metasurface are coordinated and optimized, thereby simultaneously taking into account link reliability, overall network throughput and long-term sustainable operation capability of the system.

[0008] The objective of this invention can be achieved through the following technical solutions: According to a first aspect of the present invention, a signal transmission method for an underground wireless power supply sensor network is provided, wherein a reconfigurable smart metasurface is introduced into the underground wireless power supply sensor network, and the propagation link is controlled by adjusting the phase of the reflective unit of the reconfigurable smart metasurface. The method includes: The deployment location of the reconfigurable smart metasurface is optimized offline to determine the optimal deployment location; Obtain the current network status parameters of the underground wireless power supply sensor network at the optimal deployment location. The network status parameters include the downlink status and uplink status of each underground sensor node, the auxiliary link status of the reconfigurable smart metasurface, and the energy information currently collected by the underground sensor node. Based on the current network state parameters, and under the conditions of satisfying link reliability constraints and energy causality constraints, with the joint optimization objective of improving the overall network throughput while taking into account the energy sustainability of underground sensing nodes, a pre-trained deep reinforcement learning model suitable for continuous action space is used to jointly optimize the reconfigurable intelligent metasurface phase configuration, downlink time slot allocation, and uplink transmit power allocation, so as to obtain the optimal joint control strategy for the current transmission frame and execute it. After the current transmission frame is completed, the network status parameters are updated to optimize transmission control for the next transmission frame.

[0009] Preferably, the offline optimization of the deployment location of the reconfigurable smart metasurface to determine the optimal deployment location specifically involves: For each candidate deployment location, the corresponding air segment propagation distance and underground propagation distance are calculated based on the geometric relationship between the radio frequency energy source, the ground receiver and the underground sensor node, and the downlink receiving power and uplink receiving power are obtained by combining the ground air cascade propagation model. Based on the downlink received power, uplink received power, downlink wireless power supply (WET) / backscatter communication (BSC) phase duration corresponding to the candidate deployment location, downlink time slot allocation for each underground sensor node, and uplink wireless information transmission (WIT) phase transmission parameters, the data collection energy, downlink backscatter throughput, and uplink active transmission throughput of each underground sensor node are calculated. The downlink backscatter throughput and uplink active transmission throughput of each underground sensor node are then summed to obtain the overall network throughput corresponding to the candidate deployment location. The candidate deployment location that maximizes the overall network throughput is selected as the optimal deployment location for the reconfigurable smart metasurface.

[0010] Preferably, a fixed longitudinal coordinate is used to search for the deployment location of the reconfigurable smart metasurface only in the horizontal position, and the search range for the deployment location is limited to the direction of the line connecting the radio frequency energy source and the ground receiver.

[0011] Preferably, based on the current network state parameters, and under the conditions of satisfying link reliability constraints and energy causality constraints, with the joint optimization objective of improving overall network throughput while taking into account the energy sustainability of underground sensing nodes, a pre-trained deep reinforcement learning model suitable for continuous action space is used to jointly optimize the reconfigurable intelligent metasurface phase configuration, downlink time slot allocation, and uplink transmit power allocation, to obtain and execute the optimal joint control strategy for the current transmission frame, specifically including: Joint optimization objective : , The constraints are: , , , , , , , , In the formula: For the first The active transmission throughput of each underground sensor node during the uplink phase, of which Indicates the sequential number of the underground sensor node being decoded; For the first Downlink backscattering throughput of an underground sensing node; For the first The energy collected by an underground sensing node within the current transmission frame; The number of underground sensor nodes; , and These represent the weight coefficients of the corresponding items; The phase matrix of the reconfigurable smart metasurface; Represents the reconfigurable smart metasurface. The phase of each reflecting unit; The number of reflective units for the reconfigurable smart metasurface; It is a preset positive number used to limit the minimum duration percentage of the WIT phase in uplink wireless information transmission; This represents the percentage of time spent in the downward phase. This represents the percentage of time spent in the upward phase. Assign vectors to downlink time slots for each underground sensor node, where For the first The percentage of time slots for each underground sensing node during the downlink phase; Let be the expected uplink transmit power vector of each underground sensor, where For the first The expected transmit power of an underground sensing node during the uplink phase; The length of the transmission frame; For the first The actual uplink transmission power of each underground sensing node meets the requirements. , For the first The maximum allowable uplink transmit power for each underground sensor node. Indicates the first Energy collected by each underground sensing node within the current transmission frame The proportion that can be used for uplink active wireless information transmission.

[0012] Preferably, the reinforcement learning model employs a deep reinforcement learning algorithm based on stochastic ensemble double Q-learning, and the model training process is as follows: 1) Obtain the current network state at decision time step t. And input the policy network to generate continuous action vectors; The network state is defined as follows: , ; Action space is defined as: ; In the formula: The network state at time step t; For time step t, the next step The status of each underground sensor node; For the first Small-scale fading coefficient of the downlink direct link corresponding to each underground sensor node; is the small-scale fading coefficient of the uplink direct link corresponding to the kth underground sensor node; The channel vector from the reconfigurable smart metasurface to the ground receiver is given by the superscript H, which indicates the conjugate transpose. Let be the phase matrix of the reconfigurable smart metasurface at time step t-1; For the first Channel vectors from underground sensing nodes to reconfigurable smart metasurfaces; For the first Large-scale loss of the reconfigurable smart metasurface auxiliary link corresponding to each underground sensing node; For the first The energy collected by an underground sensing node within the current transmission frame; The available energy of the k-th underground sensor node at the start of decision time step t is used to determine whether the underground sensor node meets the backscatter communication BSC's reflection circuit activation condition in its corresponding downlink sub-time slot. For the first The phase corresponding to each reflecting unit; The number of reflective units for the reconfigurable smart metasurface; This represents the percentage of time spent in the downward phase. Output the policy network for the first time The original downlink time slot allocation vector corresponding to each underground sensor node; For the first The expected transmit power of an underground sensing node during the uplink phase; The number of underground sensor nodes; 2) Execute the down-row and up-row phases based on the action vector, calculate the immediate reward, and store it in the experience replay pool; 3) Update multiple value network parameters by sampling training samples, update policy network parameters, and perform soft updates on the target value network, including: Training samples are sampled from the experience replay pool. The network states and joint control actions in each training sample are combined to form state-action pairs, which are then input into the training samples. Value network Each value network is used to estimate the state-action value corresponding to the same network state and joint control action, and different value networks have mutually independent network parameters. Every value network Each one corresponds to a target value network. When calculating the target value, from Randomly selected from a target value network A subset of target value networks is formed by several target value networks, among which < And calculate the target value based on the minimum state-action value output by each target value network in the target value network subset; The parameters of each value network are updated based on the error between the state-action value output by each value network and the target value. The parameters of the strategy network are updated based on the value estimation results of the value network. A soft update method is used to update the parameters of each target value network based on the parameters of the corresponding value network. 4) Repeat the process of state observation, action generation, environment interaction, network update, and target value network soft update. After training is completed, output the trained reinforcement learning model.

[0013] Preferably, the instant reward is calculated using the following expression: In the formula: For the first The active transmission throughput of each underground sensor node during the uplink phase, of which Indicates the sequential number of the underground sensor node being decoded; For the first Downlink backscattering throughput of an underground sensing node; , , These are the weighting coefficients for the corresponding items; Low-energy penalty weights; This is the preset low energy protection threshold.

[0014] Preferably, within each transmission frame, the downlink phase is executed first, followed by the uplink phase, specifically including: Downlink Phase: During the downlink wireless power supply (WET) / backscatter communication (BSC) phase, the RF power source continuously transmits RF signals to various underground sensor nodes. The reconfigurable smart metasurface adjusts the phase of each reflective unit through phase configuration. Downlink sub-time slots are allocated to each underground sensor node according to the downlink time slot allocation results. In each downlink sub-time slot, the underground sensor node allocated the downlink sub-time slot performs backscatter communication (BSC) when the reflection circuit activation conditions are met. The remaining underground sensor nodes perform energy acquisition operations corresponding to wireless power transmission (WET) in the sub-time slot. When the energy available for driving the reflection circuit of the underground sensor node allocated the downlink sub-time slot is lower than the reflection circuit activation energy threshold, the underground sensor node remains silent in the sub-time slot and does not perform backscatter communication. Uplink phase: Each underground sensor node uses the energy collected during the wireless energy harvesting process in the downlink wireless power supply WET / backscatter communication BSC phase to simultaneously transmit sensor information to the ground receiver; the ground receiver sorts the underground sensor nodes from highest to lowest received power and uses continuous interference cancellation to complete multi-user signal detection.

[0015] According to a second aspect of the present invention, an underground wireless power supply sensor network system is provided, the system comprising a controller, a radio frequency power source, a reconfigurable smart metasurface, a surface receiver, and multiple underground sensor nodes; The controller uses the method described above to jointly optimize the reconfigurable intelligent metasurface phase configuration, downlink time slot allocation, and uplink transmit power allocation of the current transmission frame based on the current network state parameters, and outputs the optimal joint control strategy. Based on the optimal joint control strategy output by the controller, within each transmission frame, the downlink phase is executed first, followed by the uplink phase, specifically including: Downlink Phase: During the downlink wireless power supply (WET) / backscatter communication (BSC) phase, the RF power source continuously transmits RF signals to various underground sensor nodes. The reconfigurable smart metasurface adjusts the phase of each reflective unit through phase configuration. Downlink sub-time slots are allocated to each underground sensor node according to the downlink time slot allocation results. In each downlink sub-time slot, the underground sensor node allocated the downlink sub-time slot performs backscatter communication (BSC) when the reflection circuit activation conditions are met. The remaining underground sensor nodes perform energy acquisition operations corresponding to wireless power transmission (WET) in the sub-time slot. When the energy available for driving the reflection circuit of the underground sensor node allocated the downlink sub-time slot is lower than the reflection circuit activation energy threshold, the underground sensor node remains silent in the sub-time slot and does not perform backscatter communication. Uplink Phase: Each underground sensor node utilizes the energy collected during the wireless energy harvesting process in the downlink wireless power supply (WET / backscatter communication BSC) phase to simultaneously transmit sensor information to the ground receiver. The ground receiver sorts the underground sensor nodes according to their received power from highest to lowest and uses continuous interference cancellation to complete multi-user signal detection.

[0016] According to a third aspect of the present invention, an electronic device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the program to implement any of the methods described above.

[0017] According to a fourth aspect of the invention, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements any of the methods described herein.

[0018] Compared with the prior art, the present invention has the following advantages: (1) This invention addresses the problems of large attenuation of underground links, nonlinearity of energy harvesting process, strong coupling between downlink power supply and uplink transmission, and insufficient adaptability of traditional static optimization methods in dynamic soil environments in underground wireless power supply sensor networks. It introduces reconfigurable smart metasurface technology and uses a reinforcement learning model to jointly optimize the phase configuration, downlink time slot allocation and uplink transmit power allocation of the reconfigurable smart metasurface. By coordinating the relationship between downlink wireless power transmission, backscatter communication and uplink active information transmission, it effectively improves the system time utilization, energy utilization and overall network throughput, and has good robustness, real-time performance and engineering application value.

[0019] (2) This invention adopts a deep reinforcement learning algorithm based on stochastic ensemble double Q learning. It performs parallel value estimation of the same state-action pair through multiple value networks with independent parameters, and randomly selects some target value networks for minimum value aggregation during the target value calculation process to reduce the value estimation bias of a single value network and the impact of value overestimation on policy updates. At the same time, it uses high-update data to compare environmental interaction samples for multiple parameter updates, improves the efficiency of training sample utilization, thereby improving the stability of value estimation and the convergence speed of model training in the high-dimensional continuous joint control process of RIS phase configuration, downlink time slot allocation and uplink transmit power.

[0020] (3) In the reinforcement learning instant reward, the present invention simultaneously considers the uplink active transmission throughput, downlink backscatter throughput, underground sensor node acquisition energy and low energy gap. By guiding the control strategy through acquisition energy reward and low energy gap penalty, the power supply status of low-energy underground sensor nodes is improved. While increasing the overall network throughput, the situation of underground sensor nodes being in a low energy state for a long time is reduced, thereby improving the continuous operation capability of the underground wireless power supply sensor network. Attached Figure Description

[0021] Figure 1 This is a flowchart of the method of the present invention.

[0022] Figure 2 This is a schematic diagram of the structure and transmission frame of an underground wireless power supply sensor network system based on a reconfigurable smart metasurface.

[0023] Figure 3 This is a flowchart of the method of the present invention.

[0024] Figure 4 Optimize curves for RIS deployment locations.

[0025] Figure 5 The training convergence performance of different reinforcement learning algorithms under the joint optimization problem of this invention is compared.

[0026] Figure 6 A comparison of uplink throughput for different schemes as the number of reflection units changes.

[0027] Figure 7 A comparison of uplink throughput for different schemes as the number of sensors changes.

[0028] Figure 8 A comparison of uplink throughput variations with burial depth for different schemes.

[0029] Figure 9 This study compares the uplink throughput of different schemes with changes in water content. Detailed Implementation

[0030] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0031] Reconfigurable smart metasurfaces are a technology that reconstructs the wireless propagation environment by adjusting the parameters of the reflecting elements. Specifically, after a radio frequency signal is reflected by a reconfigurable smart metasurface during propagation, its reflection phase and propagation direction can be controlled, thereby improving link quality, expanding system coverage, and increasing spectrum and energy utilization efficiency. Since signal propagation from above ground to underground and from underground to above ground in underground wireless power supply sensor networks is severely affected by soil attenuation, reconfigurable smart metasurfaces can be introduced into underground wireless power supply sensor network systems to enhance the energy harvesting capability and information transmission performance of underground nodes by strengthening the composite propagation link. Compared to traditional underground wireless power supply sensor networks, underground wireless power supply sensor networks integrating reconfigurable smart metasurface technology have significant application potential in improving downlink wireless power transmission efficiency, enhancing uplink reliability, and increasing overall network throughput.

[0032] This invention addresses the problems of high underground link attenuation, nonlinear energy harvesting, and strong coupling between downlink power supply and uplink transmission in underground wireless power supply sensor networks. It introduces reconfigurable smart metasurface technology to design a multi-user underground wireless power supply sensor network system based on reconfigurable smart metasurface. With the goal of improving the overall network throughput and ensuring the long-term sustainable operation of the system, under the premise of ensuring reliable links and energy causal constraints, it achieves high-throughput data transmission control of the network as a whole by jointly optimizing the phase configuration of the reconfigurable smart metasurface, the downlink wireless power transmission / backscattering communication process, and the uplink active wireless information transmission process.

[0033] This invention provides a signal transmission method for underground wireless power supply sensor networks. A reconfigurable smart metasurface is introduced into the underground wireless power supply sensor network, and the propagation link is controlled by adjusting the phase of the reconfigurable smart metasurface's reflective units. Figure 1 As shown, the method includes the following steps: Step S1: Perform offline optimization of the deployment location of the reconfigurable smart metasurface to determine the optimal deployment location, specifically as follows: For each candidate deployment location, the corresponding air segment propagation distance and underground propagation distance are calculated based on the geometric relationship between the radio frequency energy source, the ground receiver and the underground sensor node, and the downlink receiving power and uplink receiving power are obtained by combining the ground air cascade propagation model. Based on downlink and uplink power, a unified resource allocation evaluation criterion is used to evaluate the downlink wireless power supply (WET) / backscatter communication (BSC) phase duration, downlink time slot allocation for each underground sensor node, and uplink wireless information transmission (WIT) phase transmission parameters at candidate deployment locations, thereby obtaining the corresponding downlink backscatter throughput, uplink active transmission throughput, and overall network throughput. The candidate deployment location that maximizes the overall network throughput is selected as the optimal deployment location for the reconfigurable smart metasurface.

[0034] For any candidate deployment location of the reconfigurable smart metasurface Optimal deployment location Represented as: In the formula: This represents the set of candidate deployment locations for the reconfigurable smart metasurface; Indicates the location of a given candidate deployment. The overall network throughput of the system is calculated based on the corresponding uplink and downlink power and resource allocation parameters.

[0035] In one embodiment of the present invention, considering that the impact of large-scale path loss on the height of the reconfigurable smart metasurface is relatively weak, the longitudinal coordinate is fixed in this embodiment. =5 m, search only for horizontal positions.

[0036] In another embodiment of the present invention, with a fixed longitudinal coordinate Based solely on the horizontal position search, the search range is limited to the direction of the line connecting the radio frequency power source and the ground receiver, i.e., let =0, the optimal deployment location is obtained through offline one-dimensional search.

[0037] After determining the optimal deployment location, subsequent network status acquisition, joint optimization solutions, and transmission execution processes are carried out at that location.

[0038] Step S2: Obtain the network status parameters of the underground wireless power supply sensor network at the optimal deployment location in the previous transmission frame. The network status parameters include the downlink status and uplink status of each underground sensor node, the auxiliary link status of the reconfigurable smart metasurface, and the energy information currently collected by the underground sensor nodes.

[0039] In one embodiment of the present invention, the underground wireless power supply sensor network system based on the reconfigurable smart metasurface includes a radio frequency power source PS, a reconfigurable smart metasurface RIS, a ground receiver AP, and multiple underground sensor nodes UD. The reconfigurable smart metasurface RIS regulates the propagation links from the ground to the underground and from the underground to the ground by adjusting the phase of the reflective unit, so as to enhance the energy acquisition capability and information transmission capability of the underground nodes.

[0040] The entire transmission frame consists of two phases: a downlink phase for wireless power transfer and backscatter communication (WET / BSC) based on Time Division Multiple Access (TDMA) and an uplink phase for wireless information transmission (WIT) based on Non-Orthogonal Multiple Access (NOMA). The downlink phase is used to complete wireless power transfer and backscatter communication, while the uplink phase is used to complete the active wireless information transmission of the underground sensor node.

[0041] During the downlink phase, the RF power received by the k-th underground sensor node consists of two parts: the direct link and the RIS auxiliary link, which can be expressed as: In the formula: This refers to the transmit power of the radio frequency energy source. and These are the antenna gains for the radio frequency power source and the underground sensing node, respectively. This refers to the small-scale fading coefficient of the downlink direct link; and These are the channel vectors from the radio frequency energy source PS to the reconfigurable smart metasurface RIS and from the reconfigurable smart metasurface RIS to the underground node, respectively. The phase matrix of the reconfigurable smart metasurface RIS.

[0042] The large-scale losses of the downlink direct link and the reconfigurable smart metasurface RIS-assisted link are respectively expressed as: In the formula: Indicates air segment propagation loss, This represents the refractive loss at the air-soil interface. This represents the underground propagation loss. Furthermore, the above three items can be expressed as follows: In the formula: For carrier frequency, For the speed of light in free space, and These represent the real and imaginary parts of the complex permittivity of soil, respectively. and These are the attenuation constant and the phase shift constant, respectively.

[0043] The attenuation constant and phase shift constant can be expressed as follows: in, The relative magnetic permeability of the soil. The permeability of free space, The vacuum dielectric constant is given. The soil complex dielectric constant can be determined by the MBSDM model based on the soil clay content, volumetric water content and operating frequency, i.e. (9)-(24).

[0044] It can then be obtained that: (10) (11) (12) In the formula: n m , n d , n b , n f Represents refractive index, κ m , κ d , κ b , κ f The subscript represents the value of the normalized attenuation coefficient. m , d , b , f These represent wet soil, dry soil, bound water in soil, and free water in soil, respectively. m vt It is the maximum volume of bound water in a given soil.

[0045] The refractive index and normalized attenuation coefficient for dry soil, bound water in soil, and free water in soil are expressed as follows: (13) (14) The real and imaginary parts of the dielectric constant and loss factor for bound water and free water in the soil are derived from the Debye relaxation equation: (15) (16) In the formula: , and These are the electrical conductivity, relaxation time, and dielectric constant at the low-frequency limit, which are related to soil bound water and soil free water, respectively. The dielectric constant is 4.9 in both bound water and free water in soil, and is at the high-frequency limit.

[0046] The spectral parameters in (10)-(16) above can be obtained from a large number of soil samples, and the specific values ​​are expressed as follows: (17) (18) (19) (20) (twenty one) (twenty two) (twenty three) (twenty four) (25) During the upward phase, the first Average received power from underground sensor nodes to surface receivers Determined by both the direct path and the auxiliary path of the reconfigurable smart metasurface, it can be represented as: In the formula: For the first The actual uplink transmission power of each underground sensor node; This refers to the antenna gain at the receiving end; For the first Small-scale fading coefficient of the uplink direct link corresponding to each underground sensor node; This represents the channel vector from the underground sensing node to the reconfigurable smart metasurface. The channel vector from the reconfigurable smart metasurface to the ground receiver is given by the superscript H, which indicates the conjugate transpose.

[0047] Among them, the Large-scale loss of the uplink direct link corresponding to each underground sensing node Large-scale loss of uplink reconfigurable smart metasurface-assisted links They are represented as follows: In the formula: Indicates air segment propagation loss; Indicates underground propagation loss; This represents the air segment propagation distance from the corresponding ground-based launch point to the ground-based receiving AP; Let represent the underground propagation distance of the k-th underground sensing node propagating to the surface along the RIS direction of the reconfigurable smart metasurface; To represent the air segment propagation distance from the corresponding ground exit position to the reconfigurable intelligent metasurface RIS; .

[0048] Therefore, the network status information obtained in step S2 should at least include the downlink status, uplink status, RIS auxiliary link status, and the energy information currently collected by each underground sensor node, so as to provide input for subsequent joint optimization.

[0049] Step S3: Based on the current network state parameters, under the conditions of satisfying link reliability constraints and energy causality constraints, with the joint optimization objective of improving the overall network throughput and taking into account the energy sustainability of underground sensor nodes, a pre-trained reinforcement learning model is used to jointly optimize the reconfigurable smart metasurface phase configuration, downlink time slot allocation, and uplink transmit power allocation, so as to obtain the optimal joint control strategy for the current transmission frame and execute it.

[0050] Specifically, joint optimization objectives Represented as: The constraints are: In the formula: For the first The active transmission throughput of each underground sensor node during the uplink phase, of which Indicates the sequential number of the underground sensor node being decoded; For the first Downlink backscattering throughput of an underground sensing node; For the first The energy collected by an underground sensing node within the current transmission frame; The number of underground sensor nodes; , and These represent the weight coefficients of the corresponding items; The phase matrix of the reconfigurable smart metasurface; Represents the reconfigurable smart metasurface. The phase of each reflecting unit; The number of reflective units for the reconfigurable smart metasurface; It is a preset positive number used to limit the minimum duration percentage of the WIT phase in uplink wireless information transmission; This represents the percentage of time spent in the downward phase. This represents the percentage of time spent in the upward phase. Assign vectors to downlink time slots for each underground sensor node, where For the first The percentage of time slots for each underground sensing node during the downlink phase; Let be the expected uplink transmit power vector of each underground sensor, where For the first The expected transmit power of an underground sensing node during the uplink phase; The length of the transmission frame; For the first The actual uplink transmission power of each underground sensing node meets the requirements. , For the first The maximum allowable uplink transmit power for each underground sensor node. Indicates the first Energy collected by each underground sensing node within the current transmission frame The proportion that can be used for uplink active wireless information transmission.

[0051] The process of obtaining some constraints is as follows: The downlink time slot allocation vector needs to be projected onto the feasible region through a normalized mapping, that is: Where softmax() represents the normalization mapping function, its k-th component can be expressed as: Therefore, the time slot proportion of the k-th underground sensing node in the downlink phase is: This ensures the downlink time slots corresponding to each underground sensor node. All are non-negative and satisfy: This ensures that the downlink time slots of each underground sensor node are non-negative and that the total duration meets the constraints.

[0052] In one embodiment of the present invention, the reinforcement learning model adopts a deep reinforcement learning algorithm based on stochastic ensemble double Q learning. It is oriented towards a continuous action space and can uniformly solve the phase configuration of reconfigurable smart metasurfaces, the downlink TDMA-based WET / BSC stage time slot allocation, and the uplink NOMA-based WIT stage transmit power allocation.

[0053] This embodiment employs a deep reinforcement learning algorithm based on stochastic ensemble double Q learning. It mainly considers that the joint optimization action includes the phase of multiple reconfigurable smart metasurface reflective units, the downlink time slot allocation of multiple underground sensing nodes, and the uplink transmission power, forming a high-dimensional continuous action space. Furthermore, different control variables jointly affect the energy harvesting, backscatter communication, and uplink active transmission performance of the underground sensing nodes.

[0054] Specifically, a deep reinforcement learning algorithm based on stochastic ensemble double Q-learning sets up multiple value networks, each estimating the long-term cumulative reward corresponding to the current network state and joint control actions. When calculating the target value, a preset number of target value networks are randomly selected from these networks, and the target value corresponding to the current training sample is calculated based on the estimation results of the selected networks. The ensemble value estimation formed by multiple value networks reduces the impact of estimation errors from a single value network on policy network updates caused by limited training samples, random changes in channel states, and high action dimensions. Furthermore, by randomly selecting some target value networks for target value calculation, the accumulation of value estimation bias during continuous training is reduced, thereby improving the stability of value network parameter updates.

[0055] Furthermore, the deep reinforcement learning algorithm based on stochastic ensemble double Q-learning employs a high update-to-data ratio. After obtaining the state transition sample generated by a single environmental interaction, it performs multiple parameter updates for the value network and policy network on the training samples sampled from the experience replay pool. Thus, the same environmental interaction sample can be used for multiple parameter learning iterations, improving the utilization efficiency of training samples. In underground wireless power supply sensor networks, each environmental interaction corresponds to a complete transmission process including the downlink WET / BSC phase and the uplink WIT phase. Therefore, by increasing the utilization of single environmental interaction samples, the number of environmental interactions required to obtain a stable joint control policy can be reduced, and the model training convergence speed can be improved.

[0056] Therefore, compared to reinforcement learning methods that use a single or fewer value estimation networks or utilize the acquired samples less frequently, this embodiment combines multi-value network ensemble estimation, random subset target value calculation, and a high update data ratio to improve the stability of value estimation and sample utilization efficiency in a high-dimensional continuous action space. This enables the reinforcement learning model to more stably learn the joint control relationship between reconfigurable intelligent metasurface phase configuration, downlink time slot allocation, and uplink transmit power, thereby improving the convergence speed and control stability of the joint optimization strategy in a dynamic underground channel environment.

[0057] The model training process is as follows: 1) Obtain the current network state at decision time step t. And input the policy network to generate continuous action vectors; The network state is defined as follows: Action space is defined as: In the formula: The network state at time step t; For time step t, the next step The status of each underground sensor node; For the first Small-scale fading coefficient of the downlink direct link corresponding to each underground sensor node; is the small-scale fading coefficient of the uplink direct link corresponding to the kth underground sensor node; The channel vector from the reconfigurable smart metasurface to the ground receiver is given by the superscript H, which indicates the conjugate transpose. Let be the phase matrix of the reconfigurable smart metasurface at time step t-1; For the first Channel vectors from underground sensing nodes to reconfigurable smart metasurfaces; For the first Large-scale loss of the reconfigurable smart metasurface auxiliary link corresponding to each underground sensing node; The available energy of the k-th underground sensor node at the start of decision time step t is used to determine whether the underground sensor node meets the reflection circuit activation conditions within its corresponding BSC time slot. For the first The phase corresponding to each reflecting unit; The number of reflective units for the reconfigurable smart metasurface; This represents the percentage of time spent in the downward phase. Output the policy network for the first time The original downlink time slot allocation vector corresponding to each underground sensor node; For the first The expected transmit power of an underground sensing node during the uplink phase; This represents the number of underground sensor nodes.

[0058] 2) Execute the down-row and up-row phases based on the action vector, calculate the immediate reward, and store it in the experience replay pool; The instant reward is calculated using the following expression: (38) In the formula: For the first The active transmission throughput of each underground sensor node during the uplink phase, of which Indicates the sequential number of the underground sensor node being decoded; , , These are the weighting coefficients for the corresponding items; Low-energy penalty weights; This is the preset low energy protection threshold.

[0059] Specifically, the uplink active transmission throughput term and downlink backscatter throughput term in the instant reward are used to characterize the data transmission performance of the current transmission frame; the energy acquisition reward term is used to encourage underground sensing nodes to obtain more available energy, thereby increasing the energy available for uplink active information transmission under energy causal constraints.

[0060] Furthermore, an immediate reward system includes a low-energy gap penalty. This penalty applies when the energy collected by the k-th underground sensor node falls below a preset low-energy guarantee threshold. At that time, according to Determine the corresponding low energy gap; the larger the energy gap, the greater the reduction in immediate rewards. Reaching or exceeding At that time, the corresponding low-energy gap penalty is zero.

[0061] With the above settings, when an underground sensor node remains in a low-energy state, the joint control action that keeps the node in a low-energy state will receive a lower immediate reward. This reduces the tendency to select such joint control actions during subsequent policy network parameter updates and increases the tendency to select RIS phase configuration and downlink time slot allocation schemes that are favorable to the underground sensor node for wireless energy harvesting. Therefore, while improving overall network throughput, the probability of some underground sensor nodes remaining in a low-energy state for extended periods can be reduced, thus improving the continuous operation capability of the underground wireless power supply sensor network.

[0062] 3) Update multiple value network parameters by sampling training samples, update policy network parameters, and perform soft updates on the target value network.

[0063] 4) Repeat the processes of state observation, action generation, environment interaction, network update, and target value network soft update. After training, output the trained reinforcement learning model. The specific implementation is shown in Table 1 below.

[0064] Table 1 In addition to the aforementioned deep reinforcement learning model based on stochastic ensemble double Q-learning, this invention can also employ other deep reinforcement learning models suitable for continuous action spaces to achieve joint optimization. The different reinforcement learning models described above use the same network state definition, continuous action space, action feasible region mapping method, and immediate reward definition; the main differences lie in the different policy network and value network parameter update mechanisms.

[0065] In another embodiment of the present invention, the deep reinforcement learning model suitable for continuous action spaces employs the soft actor-critic algorithm (SAC), and the model training process specifically includes: 1) Initialize the random policy network, the first value network, the second value network, and the first target value network and the second target value network corresponding to the first value network and the second value network, respectively, and initialize the experience replay pool and temperature parameters.

[0066] 2) Obtain the current network state at decision time step t. The current network state is input into a random policy network, which generates a probability distribution corresponding to the original continuous actions, and the original continuous actions are obtained by sampling from the probability distribution. The original continuous actions are mapped according to the aforementioned feasible domain mapping method to obtain the executable joint control actions of the current transmission frame. .

[0067] 3) Execute the downlink WET / BSC phase and uplink WIT phase of the current transmission frame according to the executable joint control action, and obtain an immediate reward. and the network state corresponding to the next decision time step. and the state transition sample ( , , , Store it in the experience replay pool.

[0068] 4) Randomly sample multiple state transition samples from the experience replay pool. For the next network state... The original continuous actions are sampled by a random policy network. The next executable joint control action is obtained through feasible domain mapping. According to the first target value network and the second target value network, for ( , The smaller value from the value estimation results is used to calculate the target value, which is then combined with the immediate reward, discount factor, and entropy regularization term of the stochastic policy.

[0069] 5) Calculate the error between the value estimation results of the first value network and the second value network for the current state-action pair and the target value, respectively, and update the parameters of the first value network and the second value network according to the error.

[0070] 6) Regenerate the original continuous actions from the random policy network for the sampled network state, and obtain the corresponding executable joint control actions through feasible region mapping; update the parameters of the random policy network according to the smaller state-action value output by the first value network and the second value network and the policy entropy, so that the policy network can improve the long-term cumulative reward and maintain the exploration ability in the continuous action space.

[0071] 7) Update the temperature parameters according to the preset target entropy to adjust the weight of the policy entropy in the policy network parameter update process, so that the model can adaptively adjust the exploration degree according to the training state.

[0072] 8) Using a soft update method, update the parameters of the first target value network and the second target value network respectively based on the current parameters of the first value network and the second value network.

[0073] 9) Repeat the processes of network state acquisition, random action sampling, action feasible region mapping, environment interaction, value network update, policy network update, temperature parameter update, and target value network soft update until the preset training termination condition is met to obtain the trained SAC reinforcement learning model.

[0074] After the model training is completed, the parameters of the trained random policy network are fixed, the original continuous actions corresponding to the current transmission frame are generated according to the current network state, and the executable joint control actions are obtained through the feasible domain mapping.

[0075] In another embodiment of the present invention, the deep reinforcement learning model applicable to continuous action spaces employs the Deep Deterministic Policy Gradient Algorithm (DDPG), and the model training process specifically includes: 1) Initialize the deterministic policy network, value network, target policy network, and target value network, and initialize the experience replay pool; wherein, the initial parameters of the target policy network correspond to the initial parameters of the deterministic policy network, and the initial parameters of the target value network correspond to the initial parameters of the value network.

[0076] 2) Obtain the current network state at decision time step t. The current network state is input into the deterministic policy network, which then generates the original continuous actions corresponding to the current network state. During the training phase, random exploration perturbations are added to the original continuous actions to improve the exploration capability in the continuous action space.

[0077] 3) The original continuous actions after adding random exploration perturbation are processed according to the aforementioned feasible region mapping method to obtain executable joint control actions that satisfy the reconfigurable smart metasurface phase constraint, transmission frame duration constraint, BSC time slot allocation constraint, and energy causality constraint. .

[0078] 4) Execute the current transmission frame according to the executable joint control action to obtain an immediate reward. and the next network state and the state transition sample ( , , , Store it in the experience replay pool.

[0079] 5) Randomly sample multiple state transition samples from the experience replay pool. Then, set the next network state... Input the target policy network, which generates the next original continuous action, and obtains the next executable joint control action through the aforementioned feasible domain mapping.

[0080] 6) Input the next network state and the corresponding next executable joint control action into the target value network, and calculate the target state-action value based on the target value network output, immediate reward and discount factor.

[0081] 7) Input the current network state and the currently executable joint control action into the value network, and update the value network parameters according to the error between the value network output and the target state-action value, so that the value network learns the correspondence between the network state and the joint control action and the long-term cumulative reward.

[0082] 8) Input the current network state into the deterministic policy network, and update the parameters of the deterministic policy network according to the value evaluation result of the value network for the joint control actions generated by the deterministic policy network, so that the deterministic policy network can improve the state-action value corresponding to the generated joint control actions.

[0083] 9) Use a soft update method to update the target policy network parameters based on the current deterministic policy network parameters, and update the target value network parameters based on the current value network parameters.

[0084] 10) Repeat the process of network state acquisition, deterministic action generation, random exploration, action feasible region mapping, environment interaction, experience sampling, value network update, policy network update and target network soft update until the preset training termination condition is reached to obtain the trained DDPG reinforcement learning model.

[0085] In the actual control phase after training is completed, the deterministic policy network directly generates the original continuous actions based on the current network state, without adding the random exploration perturbations from the training phase, and obtains the executable joint control actions corresponding to the current transmission frame through the feasible region mapping.

[0086] In another embodiment of the present invention, the deep reinforcement learning model applicable to continuous action spaces employs the proximal policy optimization algorithm (PPO), and the model training process specifically includes: 1) Initialize the random policy network and the state value network. The random policy network is used to generate the probability distribution of continuous actions based on the network state, and the state value network is used to estimate the long-term cumulative reward under the corresponding network state.

[0087] 2) Obtain the current network state at decision time step t. The current network state is input into a random policy network, which generates a probability distribution corresponding to the original continuous actions, and the original continuous actions are obtained by sampling from the probability distribution. .

[0088] 3) Process the original continuous actions according to the aforementioned feasible region mapping method to obtain executable joint control actions that satisfy all control constraints. And execute the current transmission frame according to the executable joint control action to obtain an immediate reward. and the next network state .

[0089] 4) Continuously collect network states, original continuous actions, executable joint control actions, immediate rewards, and the next network state corresponding to multiple decision time steps to form a batch of online training samples corresponding to the current policy network.

[0090] 5) Calculate the cumulative reward for each decision time step based on the immediate reward of each training sample and the value estimation result of the state value network, and determine the advantage value based on the difference between the cumulative reward and the output of the state value network. The advantage value is used to characterize the superiority or inferiority of the current joint control action relative to the average control effect under the current network state.

[0091] 6) Save the network parameters of the policy before update used when collecting the current batch of training samples. For the original continuous actions in the training samples, calculate the probability of the current policy network to be updated and the policy network before update generating the original continuous actions in the corresponding network states, and construct the policy update target based on the probability ratio of the two.

[0092] 7) Set a preset cutoff range for the probability ratio. When the change of the current policy network relative to the policy network before the update exceeds the preset cutoff range, limit the policy update magnitude and update the random policy network parameters in combination with the advantage value to limit the continuous joint control policy from changing too much during a single policy update.

[0093] 8) Update the state value network parameters based on the error between the cumulative reward corresponding to the training sample and the output of the state value network to improve the accuracy of the state value network in estimating the long-term cumulative reward of each network state.

[0094] 9) After performing a preset number of random policy network and state value network parameter updates on the current batch of training samples, use the updated random policy network to interact with the underground wireless power supply sensor network environment again to obtain new online training samples, and repeat the process of network state acquisition, continuous action sampling, action feasible region mapping, training sample collection, advantage value calculation, restricted policy update and state value network update.

[0095] 10) When the preset training termination condition is reached, the trained PPO reinforcement learning model is obtained; the trained random policy network is used to generate the original continuous actions according to the current network state, and the executable joint control action corresponding to the current transmission frame is obtained through the feasible domain mapping.

[0096] The aforementioned REDQ, SAC, DDPG, and PPO are all used to generate reconfigurable smart metasurface phase configuration, downlink WET / BSC phase resource allocation, and uplink WIT phase transmit power corresponding to continuous joint control actions based on the current network state, and obtain executable joint control actions that satisfy phase, time allocation, and energy causality constraints through the aforementioned feasible domain mapping.

[0097] Step S4: After the current transmission frame is completed, update the network state parameters and perform optimized transmission control for the next transmission frame. Specifically, the available energy of the k-th underground sensor node for driving the reflection circuit at the start of the next decision time step is updated as follows: .

[0098] Specifically, within each transmission frame, the downlink phase is executed first, followed by the uplink phase, which includes: Downlink Phase: During the downlink WET / BSC phase, the RF power source continuously transmits RF signals to various underground sensor nodes. The reconfigurable smart metasurface adjusts the phase of each reflective unit through phase configuration. Downlink sub-time slots are allocated to each underground sensor node sequentially according to the downlink time slot allocation results. Within each downlink sub-time slot, the underground sensor node allocated that sub-time slot performs backscatter communication (BSC) when the reflection circuit activation conditions are met. The remaining underground sensor nodes perform energy harvesting operations corresponding to wireless power transmission (WET) within the sub-time slot. When the energy available to drive the reflection circuit for an underground sensor node allocated that sub-time slot is lower than the reflection circuit activation energy threshold, the underground sensor node remains silent within the sub-time slot and does not perform backscatter communication.

[0099] Uplink phase: Each underground sensor node uses the energy collected during the wireless energy harvesting process in the aforementioned downlink WET / BSC phase to simultaneously transmit sensor information to the ground receiver; the ground receiver sorts the underground sensor nodes from highest to lowest received power and uses continuous interference cancellation to complete multi-user signal detection.

[0100] Considering the nonlinear energy harvesting characteristics of the actual rectifier circuit, the harvesting power of the k-th underground sensing node is expressed using the logistic nonlinear energy harvesting model as follows: in, Indicates the maximum sampling power. and These are the parameters for the energy harvesting circuit.

[0101] Correspondingly, during the downlink WET / BSC phase, the k-th underground sensor node performs backscatter communication within its allocated BSC time slot and wireless energy harvesting within the BSC time slots allocated to other underground sensor nodes. Since the total duration of the downlink WET / BSC phase accounts for... The proportion of BSC time slots corresponding to the k-th underground sensor node is Therefore, the effective wireless energy harvesting duration of the k-th underground sensor node within the current transmission frame is ( ) The energy it collects can be expressed as: in, The energy collected by the k-th underground sensor node in the downlink WET / BSC phase of the current transmission frame; T represents the total duration of the downlink WET / BSC phase; T is the downlink sub-slot duration allocated to the k-th underground sensing node; T represents the effective duration of wireless energy harvesting by the kth underground sensor node; T is the transmission frame length. For the downlink received power of the k-th underground sensor node The corresponding energy harvesting power.

[0102] During the downlink phase, each underground sensor node sequentially performs backscatter communication or wireless energy harvesting according to the time slot allocation. When the k-th underground sensor node enters backscatter mode, its reflection circuit must meet the activation energy threshold. Under these conditions, the received power resulting from backscattering can be expressed as: in, Let be the reflection coefficient of the k-th underground sensor node.

[0103] The corresponding backscattering signal-to-noise ratio is: Therefore, the backscattering throughput of the k-th underground sensing node during the downlink phase can be expressed as: In the formula: BW is the system bandwidth. For receiver noise power, For indicator functions; Set the energy threshold for the reflection circuit; This represents the minimum signal-to-noise ratio threshold for the backscattering link. When the energy currently available to drive the reflective circuit at the k-th underground sensing node reaches or exceeds the reflective circuit activation energy threshold... Furthermore, the signal-to-noise ratio of its backscattering link reaches or exceeds the minimum signal-to-noise ratio threshold of the backscattering link. When the effective backscattering throughput is 0, the corresponding effective backscattering throughput is calculated according to the above formula; otherwise, its effective backscattering throughput is 0.

[0104] During the downlink WET / BSC phase, each underground sensor node sequentially obtains its corresponding BSC time slot according to the time slot allocation results. For the k-th underground sensor node, within the BSC time slots corresponding to other underground sensor nodes, the k-th underground sensor node receives the radio frequency signal transmitted by the radio frequency energy source and performs wireless energy acquisition; within the BSC time slot allocated to the k-th underground sensor node itself, it determines whether to perform backscatter communication based on the energy currently available for driving the reflection circuit. When the available energy reaches or exceeds the reflection circuit activation energy threshold, the k-th underground sensor node performs backscatter communication; when the available energy is lower than the reflection circuit activation energy threshold, the k-th underground sensor node remains silent within the corresponding BSC time slot and does not perform backscatter communication.

[0105] During the uplink NOMA-based WIT phase, each underground sensor node simultaneously transmits sensor information to the ground receiver using the energy collected during the downlink phase. Since the uplink transmit power of each node is limited by energy causality constraints, the actual uplink transmit power of the k-th underground sensor node is... Determined by both the desired transmit power and the collected energy, it can be expressed as: At the ground receiving end, the sensors are first sorted from highest to lowest receiving power, and multi-user detection is performed using a continuous interference cancellation method.

[0106] If the first The physical node number corresponding to each decoded node is: , Let the sequence number of the underground sensor node being decoded represent the signal-to-interference-plus-noise ratio (SIR) as follows: Therefore, the first The active transmission throughput of an underground sensor node during the uplink phase can be expressed as: Ultimately, the overall network throughput of the system within a transmission frame can be expressed as: Therefore, by implementing the obtained optimal control scheme, the coordinated optimization between downlink wireless power transmission, backscatter communication and uplink active transmission can be achieved within the same transmission frame, thereby improving the overall network throughput.

[0107] This embodiment also provides an underground wireless power supply sensor network system, which includes a controller, a radio frequency power source, a reconfigurable smart metasurface, a ground receiver, and multiple underground sensor nodes. Based on the current network state parameters, the controller jointly optimizes and outputs the reconfigurable intelligent metasurface phase configuration, downlink time slot allocation, and uplink transmit power allocation for the current transmission frame; Based on the optimal joint control strategy output by the controller, within each transmission frame, the downlink phase is executed first, followed by the uplink phase, specifically including: Downlink Phase: During the downlink WET / BSC phase, the RF power source continuously transmits RF signals to various underground sensor nodes. The reconfigurable smart metasurface adjusts the phase of each reflective unit through phase configuration. Downlink sub-time slots are allocated to each underground sensor node sequentially according to the downlink time slot allocation results. Within each downlink sub-time slot, the underground sensor node allocated that sub-time slot performs backscatter communication (BSC) when the reflection circuit activation conditions are met. The remaining underground sensor nodes perform energy harvesting operations corresponding to wireless power transmission (WET) within the sub-time slot. When the energy available to drive the reflection circuit for an underground sensor node allocated that sub-time slot is lower than the reflection circuit activation energy threshold, the underground sensor node remains silent within the sub-time slot and does not perform backscatter communication.

[0108] Uplink Phase: Each underground sensor node utilizes the energy collected during the wireless energy harvesting process in the aforementioned downlink WET / BSC phase to simultaneously transmit sensor information to the ground-based receiver. The ground-based receiver sorts the underground sensor nodes according to their received power from highest to lowest and employs continuous interference cancellation to complete multi-user signal detection. Next, numerical simulations are used to quantitatively analyze the actual performance of the method of this invention in an underground wireless power supply sensor network scenario, and a comparison is made with four benchmark schemes to verify the effectiveness of this invention.

[0109] Simulation Setup: In this specific implementation, an underground wireless power supply sensor network is considered in an open farmland scenario. Underground sensor nodes are randomly distributed within a 100 m × 100 m × 100 m monitoring area. The system includes one radio frequency power source, one above-ground receiver, and one reconfigurable smart metasurface. The number of underground sensor nodes is set to 6 by default, with a burial depth of 0.6 m; the height of the reconfigurable smart metasurface is set to 5 m; the coordinates of the radio frequency power source are set to (-5,0,0), and the coordinates of the receiver are set to (50,0,0). To ensure that the simulation results reflect the real underground propagation environment, the soil volumetric water content is set to 5%, the soil clay content is set to 15%, the carrier frequency is set to 915 MHz, and the Rayleigh CN(0,1) model is used for small-scale fading in the air domain. The number of reflective elements of the reconfigurable smart metasurface is set to 8 to 160. The downlink backscattering reflection coefficient is set to 0.6, the energy acquisition model adopts a nonlinear logistic model, the maximum acquisition power is 9.2 μW, and the relevant parameters are set to 150 and 0.0014. The system bandwidth was set to 125 kHz, the receiver noise power to -117 dBm, the RF power source transmit power to 30 dBm, and the single transmission frame length to 1 ms. As a preferred setting, the reconfigurable smart metasurface position was first determined through an offline search method and finally fixed at (42.5, 0, 5) to avoid the influence of geometric deployment differences on subsequent comparison results. Specific simulation parameters are shown in Table 1.

[0110] Table 1 The parameters in the table are all taken from the system configuration and training settings of the current embodiment.

[0111] The four benchmark schemes are set as follows: (1) WPUSNs (no-RIS, HTT-only): No RIS, no backscattering, only the traditional energy harvesting and transmission method; (2) BSC-WPUSNs (no-RIS, BSC+HTT): No RIS, but the underground sensor node performs backscatter communication in the downlink phase and active transmission in the uplink phase; (3) Random RIS phases: RIS is used, but the RIS phases are randomly generated in [0, 2π), and only the time allocation is optimized; (4) The RIS-WPBUSN method: The existing RIS-assisted wireless backscattering underground sensor network method is adopted. Under the condition of perfect CSI estimation, the RIS phase and BC, WET and WIT time allocation are jointly optimized. Under the premise of considering the saturated nonlinear energy harvesting model, time constraints and demodulation SNR threshold constraints, the total network throughput is maximized.

[0112] Simulation Results: As a preferred setting, the deployment location of the reconfigurable smart metasurface is first optimized offline before reinforcement learning training and testing. Since the large-scale path loss is weakly related to the height of the reconfigurable smart metasurface, this embodiment fixes zRIS=5m and only searches horizontal locations. For ease of reproduction, the search range is limited to the direction of the line connecting the RF power source and the ground receiver, i.e., yRIS=0, and a one-dimensional traversal is performed on the candidate locations to calculate the overall network throughput of the system under different deployment locations. Figure 4 The optimized deployment locations for reconfigurable smart metasurfaces are presented. Figure 4 As can be seen, the system throughput exhibits a clear single-peak trend as the distance between the reconfigurable smart metasurface and the ground receiver changes, while the acquisition power shows a decreasing trend, reflecting the trade-off between downlink WET / BSC and uplink WIT. The optimal deployment location for the reconfigurable smart metasurface, maximizing overall network throughput, was ultimately selected and fixed at (42.5, 0, 5). Subsequent control strategy training and performance testing were then conducted based on this location to avoid the impact of geometric deployment differences on the algorithm comparison results.

[0113] First, the training convergence performance of different reinforcement learning algorithms is compared under the joint optimization problem of this invention. For example... Figure 5As shown, under the same system environment and training parameters, REDQ, SAC, DDPG, and PPO all gradually increase the cumulative reward during the training process and eventually stabilize. Among them, REDQ exhibits a faster reward convergence speed and reaches a higher stable reward level. This result corresponds to the internal mechanism of REDQ, which employs multiple value networks, random subset target value estimation, and a high update data ratio, indicating that it is suitable for the high-dimensional continuous joint control problem of RIS phase configuration, downlink time slot allocation, and uplink transmit power allocation involved in this invention.

[0114] like Figure 6 As shown, with the increase of the number of RIS reflector units M, the uplink throughput of all RIS-assisted schemes shows an increasing trend. When M=160, the RIS+REDQ method proposed in this invention can reach approximately 2.81 Mbps, while the existing RIS-WPBUSN method is approximately 2.45 Mbps, the random phase RIS method is approximately 1.69 Mbps, and the two benchmark schemes without RIS are only approximately 0.84–1.03 Mbps. Therefore, the method of this invention improves the throughput by approximately 15% compared to the existing RIS-WPBUSN method and by approximately 66% compared to the random phase RIS method, indicating that this invention can more effectively achieve RIS beam control and resource joint optimization.

[0115] Secondly, such as Figure 7 As shown, when the number of underground sensor nodes K increases, the throughput of the method of this invention first rises rapidly and then gradually stabilizes. When K=200, the throughput of the method of this invention stabilizes at approximately 2.70 Mbps, while the existing RIS-WPBUSN method is approximately 2.37 Mbps, the random phase RIS method is approximately 1.86 Mbps, and the two benchmark schemes without RIS are approximately 1.29–1.42 Mbps. This demonstrates that the present invention maintains a high throughput even when the network scales up, exhibiting good scalability and robustness.

[0116] Again, such as Figure 8 and Figure 9As shown, when the burial depth increases from 0.4 m to 0.8 m, the throughput of all schemes decreases due to enhanced underground attenuation, but the method of this invention still maintains a throughput of approximately 2.11 Mbps, up from approximately 3.00 Mbps. Furthermore, at a burial depth of 0.8 m, the throughput of the RIS-WPBUSN method is approximately 1.86 Mbps and the random phase RIS method is approximately 1.38 Mbps. When the soil volumetric water content increases from 5% to 15%, the throughput of the method of this invention decreases from approximately 2.50 Mbps to approximately 2.17 Mbps, but still remains at the highest level. This result demonstrates that the present invention has good adaptability to changes in soil conditions. In summary, this embodiment shows that by introducing a reconfigurable smart metasurface into an underground wireless power supply sensor network and employing the REDQ reinforcement learning algorithm to uniformly optimize RIS phase configuration, downlink TDMA-based WET / BSC time slot allocation, and uplink NOMA-based WIT transmit power allocation, the present invention can effectively improve the overall network throughput while ensuring low-energy node protection and long-term sustainable system operation, exhibiting good environmental adaptability and engineering application value.

[0117] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A signal transmission method for an underground wireless power supply sensor network, characterized in that, The underground wireless power supply sensor network introduces a reconfigurable smart metasurface, and the propagation link is controlled by adjusting the phase of the reflective unit of the reconfigurable smart metasurface. The method includes: The deployment location of the reconfigurable smart metasurface is optimized offline to determine the optimal deployment location; Obtain the current network status parameters of the underground wireless power supply sensor network at the optimal deployment location. The network status parameters include the downlink status and uplink status of each underground sensor node, the auxiliary link status of the reconfigurable smart metasurface, and the energy information currently collected by the underground sensor node. Based on the current network state parameters, and under the conditions of satisfying link reliability constraints and energy causality constraints, with the joint optimization objective of improving the overall network throughput while taking into account the energy sustainability of underground sensing nodes, a pre-trained deep reinforcement learning model suitable for continuous action space is used to jointly optimize the reconfigurable intelligent metasurface phase configuration, downlink time slot allocation, and uplink transmit power allocation, so as to obtain the optimal joint control strategy for the current transmission frame and execute it. After the current transmission frame is completed, the network status parameters are updated to optimize transmission control for the next transmission frame.

2. The signal transmission method for an underground wireless power supply sensor network according to claim 1, characterized in that, The offline optimization of the deployment location of the reconfigurable smart metasurface to determine the optimal deployment location specifically involves: For each candidate deployment location, the corresponding air segment propagation distance and underground propagation distance are calculated based on the geometric relationship between the radio frequency energy source, the ground receiver and the underground sensor node, and the downlink receiving power and uplink receiving power are obtained by combining the ground air cascade propagation model. Based on the downlink received power, uplink received power, downlink wireless power supply (WET) / backscatter communication (BSC) phase duration corresponding to the candidate deployment location, downlink time slot allocation for each underground sensor node, and uplink wireless information transmission (WIT) phase transmission parameters, the data collection energy, downlink backscatter throughput, and uplink active transmission throughput of each underground sensor node are calculated. The downlink backscatter throughput and uplink active transmission throughput of each underground sensor node are then summed to obtain the overall network throughput corresponding to the candidate deployment location. The candidate deployment location that maximizes the overall network throughput is selected as the optimal deployment location for the reconfigurable smart metasurface.

3. The signal transmission method for an underground wireless power supply sensor network according to claim 2, characterized in that, The deployment location of the reconfigurable smart metasurface is searched using a fixed vertical coordinate system and only in the horizontal position, and the search range is limited to the direction of the line connecting the radio frequency power source and the ground receiver.

4. The signal transmission method for an underground wireless power supply sensor network according to claim 1, characterized in that, Based on the current network state parameters, and under the conditions of satisfying link reliability constraints and energy causality constraints, the joint optimization objective is to improve the overall network throughput while taking into account the energy sustainability of underground sensing nodes. A pre-trained deep reinforcement learning model suitable for continuous action spaces is used to jointly optimize the reconfigurable intelligent metasurface phase configuration, downlink time slot allocation, and uplink transmit power allocation. This yields the optimal joint control strategy for the current transmission frame and executes it, specifically including: Joint optimization objective : ， The constraints are: , , , , , , , , In the formula: For the first The active transmission throughput of each underground sensor node during the uplink phase, of which Indicates the sequential number of the underground sensor node being decoded; For the first Downlink backscattering throughput of an underground sensing node; For the first The energy collected by an underground sensing node within the current transmission frame; The number of underground sensor nodes; , and These represent the weight coefficients of the corresponding items; The phase matrix of the reconfigurable smart metasurface; Represents the reconfigurable smart metasurface. The phase of each reflecting unit; The number of reflective units for the reconfigurable smart metasurface; It is a preset positive number used to limit the minimum duration percentage of the WIT phase in uplink wireless information transmission; This represents the percentage of time spent in the downward phase. This represents the percentage of time spent in the upward phase. Assign vectors to downlink time slots for each underground sensor node, where For the first The percentage of time slots for each underground sensing node during the downlink phase; Let be the expected uplink transmit power vector of each underground sensor, where For the first The expected transmit power of an underground sensing node during the uplink phase; The length of the transmission frame; For the first The actual uplink transmission power of each underground sensing node meets the requirements. , For the first The maximum allowable uplink transmit power for each underground sensor node. Indicates the first Energy collected by each underground sensing node within the current transmission frame The proportion that can be used for uplink active wireless information transmission.

5. A signal transmission method for an underground wireless power supply sensor network according to claim 1, characterized in that, The reinforcement learning model employs a deep reinforcement learning algorithm based on stochastic ensemble double Q-learning. The specific training process of the model is as follows: 1) Obtain the current network state at decision time step t. And input the policy network to generate continuous action vectors; The network state is defined as follows: ， ； Action space is defined as: ； In the formula: The network state at time step t; For time step t, the next step The status of each underground sensor node; For the first Small-scale fading coefficient of the downlink direct link corresponding to each underground sensor node; is the small-scale fading coefficient of the uplink direct link corresponding to the kth underground sensor node; The channel vector from the reconfigurable smart metasurface to the ground receiver is given by the superscript H, which indicates the conjugate transpose. Let be the phase matrix of the reconfigurable smart metasurface at time step t-1; For the first Channel vectors from underground sensing nodes to reconfigurable smart metasurfaces; For the first Large-scale loss of the reconfigurable smart metasurface auxiliary link corresponding to each underground sensing node; For the first The energy collected by an underground sensing node within the current transmission frame; The available energy of the k-th underground sensor node at the start of decision time step t is used to determine whether the underground sensor node meets the backscatter communication BSC's reflection circuit activation condition in its corresponding downlink sub-time slot. For the first The phase corresponding to each reflecting unit; The number of reflective units for the reconfigurable smart metasurface; This represents the percentage of time spent in the downward phase. Output the policy network. The original downlink time slot allocation vector corresponding to each underground sensor node; For the first The expected transmit power of an underground sensing node during the uplink phase; The number of underground sensor nodes; 2) Execute the down-row and up-row phases based on the action vector, calculate the immediate reward, and store it in the experience replay pool; 3) Update multiple value network parameters by sampling training samples, update policy network parameters, and perform soft updates on the target value network, including: Training samples are sampled from the experience replay pool. The network states and joint control actions in each training sample are combined to form state-action pairs, which are then input into the training samples. Value network Each value network is used to estimate the state-action value corresponding to the same network state and joint control action, and different value networks have mutually independent network parameters. Every value network Each one corresponds to a target value network. When calculating the target value, from Randomly selected from a target value network A subset of target value networks is formed by several target value networks, among which < And calculate the target value based on the minimum state-action value output by each target value network in the target value network subset; The parameters of each value network are updated based on the error between the state-action value output by each value network and the target value. The parameters of the strategy network are updated based on the value estimation results of the value network. A soft update method is used to update the parameters of each target value network based on the parameters of the corresponding value network. 4) Repeat the process of state observation, action generation, environment interaction, network update, and target value network soft update. After training is completed, output the trained reinforcement learning model.

6. A signal transmission method for an underground wireless power supply sensor network according to claim 5, characterized in that, The instant reward is calculated using the following expression: In the formula: For the first The active transmission throughput of each underground sensor node during the uplink phase, of which Indicates the sequential number of the underground sensor node being decoded; For the first Downlink backscattering throughput of an underground sensing node; , , These are the weighting coefficients for the corresponding items; Low-energy penalty weights; This is the preset low energy protection threshold.

7. A signal transmission method for an underground wireless power supply sensor network according to claim 1, characterized in that, Within each transmission frame, the downlink phase is executed first, followed by the uplink phase, which specifically includes: Downlink Phase: During the downlink wireless power supply (WET) / backscatter communication (BSC) phase, the RF power source continuously transmits RF signals to various underground sensor nodes. The reconfigurable smart metasurface adjusts the phase of each reflective unit through phase configuration. Downlink sub-time slots are allocated to each underground sensor node according to the downlink time slot allocation results. In each downlink sub-time slot, the underground sensor node allocated the downlink sub-time slot performs backscatter communication (BSC) when the reflection circuit activation conditions are met. The remaining underground sensor nodes perform energy acquisition operations corresponding to wireless power transmission (WET) in the sub-time slot. When the energy available for driving the reflection circuit of the underground sensor node allocated the downlink sub-time slot is lower than the reflection circuit activation energy threshold, the underground sensor node remains silent in the sub-time slot and does not perform backscatter communication. Uplink phase: Each underground sensor node uses the energy collected during the wireless energy harvesting process in the downlink wireless power supply WET / backscatter communication BSC phase to simultaneously transmit sensor information to the ground receiver; the ground receiver sorts the underground sensor nodes from highest to lowest received power and uses continuous interference cancellation to complete multi-user signal detection.

8. An underground wireless power supply sensor network system, characterized in that, The system includes a controller, a radio frequency power source, a reconfigurable smart metasurface, an above-ground receiver, and multiple underground sensing nodes. The controller applies the method described in any one of claims 1 to 7, and based on the current network state parameters, performs joint optimization of the reconfigurable intelligent metasurface phase configuration, downlink time slot allocation, and uplink transmit power allocation of the current transmission frame, and outputs the optimal joint control strategy. Based on the optimal joint control strategy output by the controller, within each transmission frame, the downlink phase is executed first, followed by the uplink phase, specifically including: Downlink Phase: During the downlink wireless power supply (WET) / backscatter communication (BSC) phase, the RF power source continuously transmits RF signals to various underground sensor nodes. The reconfigurable smart metasurface adjusts the phase of each reflective unit through phase configuration. Downlink sub-time slots are allocated to each underground sensor node according to the downlink time slot allocation results. In each downlink sub-time slot, the underground sensor node allocated the downlink sub-time slot performs backscatter communication (BSC) when the reflection circuit activation conditions are met. The remaining underground sensor nodes perform energy acquisition operations corresponding to wireless power transmission (WET) in the sub-time slot. When the energy available for driving the reflection circuit of the underground sensor node allocated the downlink sub-time slot is lower than the reflection circuit activation energy threshold, the underground sensor node remains silent in the sub-time slot and does not perform backscatter communication. Uplink phase: Each underground sensor node uses the energy collected during the wireless energy harvesting process in the downlink wireless power supply WET / backscatter communication BSC phase to simultaneously transmit sensor information to the ground receiver; the ground receiver sorts the underground sensor nodes from highest to lowest received power and uses continuous interference cancellation to complete multi-user signal detection.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1 to 7.

Images