Simultaneous transmission and reflection subarray reflector, parameter optimization methods for communication systems, and related equipment.

Abstract

The simultaneous transmission and reflection subarray reflector, the parameter optimization method for the communication system, and related equipment proposed in this application embodiment include: an energy harvesting circuit, a battery unit, and multiple subarrays. The energy harvesting circuit is connected to the battery unit to harvest ambient energy and store it in the battery unit to power the operation of the multiple subarrays. Each subarray includes a mode switching switch, a shared phase shifting circuit, a shared power distribution circuit, a shared amplification circuit, and multiple reflection and transmission units. In each subarray, all reflection and transmission units are connected to the shared phase shifting circuit, and the mode switching switch is connected to the shared amplification circuit and the shared power distribution circuit, respectively. The simultaneous transmission and reflection subarray reflector is used to control the closing state of the mode switching switch based on the mode control signal, effectively expanding the spatial coverage of the communication system, improving deployment flexibility, and enhancing the long-term average throughput and communication reliability of the communication system.

Description

Technical Field

[0001] This application relates to the field of wireless communication technology, and in particular to a method for optimizing the parameters of a simultaneous transmission and reflection subarray reflector and a communication system, as well as related equipment. Background Technology

[0002] In wireless communication systems, self-powered reconfigurable smart surface (RIS) technology based on subarray architecture can effectively reduce system power consumption and improve deployment flexibility. However, existing RIS can usually only reflect signals, that is, it can only serve users located on the same side. This unidirectional signal processing method is difficult to take into account the communication needs of users located on the other side, resulting in certain limitations in terms of spatial coverage and deployment flexibility. Summary of the Invention

[0003] This application provides a method and related equipment for optimizing the parameters of a simultaneous transmission and reflection subarray reflector and a communication system, which can solve the above-mentioned technical problems.

[0004] To achieve the above objectives, a first aspect of this application provides a simultaneous transmission and reflection subarray reflector surface, comprising: The system includes an energy harvesting circuit, a battery cell, and multiple subarrays. The energy harvesting circuit is connected to the battery cell and is used to harvest ambient energy and store it in the battery cell to power the multiple subarrays. Each of the subarrays includes a mode switching switch, a shared phase shifting circuit, a shared power distribution circuit, a shared amplification circuit, and multiple reflection and transmission units; In each of the subarrays, all the reflection and transmission units are connected to the shared phase-shifting circuit, and the mode switching switch is connected to the shared amplification circuit and the shared power distribution circuit respectively. The shared power distribution circuit and the shared phase-shifting circuit work together to achieve transmission and reflection modulation of the incident signal. When the simultaneous transmission and reflection subarray receives a mode control signal, the simultaneous transmission and reflection subarray is used to control the closing state of the corresponding mode switching switch based on the mode control signal, so as to control the corresponding reflection and transmission unit to switch between different working modes, and correspondingly change the communication link between the transmission modulation and the reflection modulation.

[0005] To achieve the above objectives, a second aspect of this application proposes a method for optimizing transmission parameters of a communication system. The communication system includes a base station, a plurality of first user terminals located in a reflection region, a plurality of second user terminals located in a transmission region, and a simultaneous transmission and reflection subarray as described in the first aspect. The method includes: The system acquires the first channel state information between each first user terminal and the base station, the second channel state information between each second user terminal and the base station, the third channel state information between the simultaneous transmission and reflection subarray reflector and the base station, and the fourth channel state information between each first user terminal and the second user terminal and the simultaneous transmission and reflection subarray reflector in the current time frame; and acquires the battery energy state information of the simultaneous transmission and reflection subarray reflector in the current time frame. The first channel state information, the second channel state information, the third channel state information, the fourth channel state information, and the battery energy state information are input into the transmission parameter optimization model for data processing to obtain optimization parameters. The optimization parameters include an optimization mode switching sequence, an optimization power allocation coefficient vector sequence, an optimization reflection phase shift sequence, an optimization transmission phase shift sequence, an optimization amplification coefficient sequence, an optimization reflection transmission power, and an optimization transmission transmission power. Based on the optimization parameters, the first channel state information, the second channel state information, the third channel state information, and the fourth channel state information, the optimized reflection reception beamforming and the optimized transmission reception beamforming of the base station are obtained. Based on the optimized mode switch sequence, the closed state of the mode switching switch of the corresponding subarray is controlled. Based on the optimized reflection phase shift sequence, the optimized transmission phase shift sequence, the optimized power distribution coefficient vector sequence, and the optimized amplification coefficient sequence, the corresponding shared phase shift circuit, the shared power distribution circuit, and the shared amplification circuit are controlled to coordinately regulate the transmission and reflection regulation of the reflective surface of the simultaneous transmission and reflection subarray. Based on the optimized reflection transmit power, the first user terminal is controlled to send first data to the base station; based on the optimized transmission transmit power, the second user terminal is controlled to send second data to the base station; based on the optimized reflection receive beamforming, the base station is controlled to receive the first data; and based on the optimized transmission receive beamforming, the base station is controlled to receive the second data.

[0006] To achieve the above objectives, a third aspect of this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the transmission parameter optimization method for the communication system as described in the second aspect.

[0007] To achieve the above objectives, a fourth aspect of the present application provides a storage medium, which is a computer-readable storage medium storing a computer program that, when executed by a processor, implements the transmission parameter optimization method for the communication system as described in the second aspect.

[0008] The simultaneous transmission and reflection subarray reflector, communication system parameter optimization method, and related equipment proposed in this application embodiment include: an energy harvesting circuit, a battery unit, and multiple subarrays. The energy harvesting circuit is connected to the battery unit to harvest ambient energy and store it in the battery unit to power the operation of the multiple subarrays. Each subarray includes a mode switching switch, a shared phase shifting circuit, a shared power distribution circuit, a shared amplification circuit, and multiple reflection and transmission units. In each subarray, all reflection and transmission units are connected to the shared phase shifting circuit, and the mode switching switch is connected to the shared amplification circuit and the shared power distribution circuit, respectively. The shared power distribution circuit and the shared phase shifting circuit work together to achieve transmission and reflection modulation of the incident signal. When the simultaneous transmission and reflection subarray reflector receives a mode control signal, it controls the closing state of the corresponding mode switching switch based on the mode control signal to control the corresponding reflection and transmission units to switch between different operating modes and correspondingly change the communication link for transmission and reflection modulation. This application embodiment achieves simultaneous transmission and reflection modulation of incident signals by introducing a shared power distribution circuit and a shared phase shifting circuit in the sub-array. This overcomes the spatial limitation of traditional unidirectional reflective surfaces that can only serve users on the same side, effectively expanding the spatial coverage of the communication system and improving deployment flexibility. Furthermore, by adopting a sub-array architecture, multiple reflection and transmission units share amplification, power distribution, and phase shifting circuits, significantly reducing the number of hardware components and the static power consumption of the basic circuit. At the same time, combined with a mode switching switch, each sub-array can flexibly switch between different operating modes based on the received control signals. Thus, in self-powered scenarios that rely on environmental energy harvesting, the system can perform finer-grained scheduling and efficient utilization of limited energy resources based on dynamically changing battery energy states. This effectively alleviates the problem of shortened effective working time caused by insufficient or fluctuating environmental energy supply, thereby significantly improving the long-term average throughput and communication reliability of the communication system while reducing overall system power consumption.

[0009] Other features and advantages of this application will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the application. The objectives and other advantages of this application may be realized and obtained by means of the structures particularly pointed out in the description, claims and drawings. Attached Figure Description

[0010] Figure 1 This is a schematic diagram of the structure of a simultaneous transmission and reflection subarray reflector provided in an embodiment of this application.

[0011] Figure 2This is a schematic diagram of another simultaneous transmission and reflection subarray reflector provided in another embodiment of this application.

[0012] Figure 3 This is a schematic diagram of the structure of a communication system provided in another embodiment of this application.

[0013] Figure 4 This is a flowchart of a method for optimizing transmission parameters of a communication system according to another embodiment of this application.

[0014] Figure 5 This is a schematic diagram of the model framework of a Hybrid Proximal Policy Optimization (HPPO) algorithm based on action combination, provided in another embodiment of this application.

[0015] Figure 6 This is a schematic diagram of a simulation structure of a communication system provided in another embodiment of this application.

[0016] Figure 7 This is a first performance simulation diagram of a parameter optimization method for a communication system provided in another embodiment of this application.

[0017] Figure 8 This is a second performance simulation diagram of a parameter optimization method for a communication system provided in another embodiment of this application.

[0018] Figure 9 This is a third performance simulation diagram of the parameter optimization method for a communication system provided in another embodiment of this application.

[0019] Figure 10 This is a schematic diagram of the hardware structure of an electronic device provided in another embodiment of this application. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0021] It should be noted that although functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart.

[0022] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0023] In this application, vectors are represented by bold lowercase letters, and matrices are represented by bold uppercase and lowercase letters. The dimension is A complex matrix. and Represents the transpose and conjugate transpose of a vector or matrix. Symbol It represents the statistical expectation. This indicates a diagonalization operation. The dimension is The identity matrix. and These represent the norm of the matrix and the modulus of the complex number, respectively.

[0024] In wireless communication systems, self-powered reconfigurable smart surface (RIS) technology based on subarray architecture can effectively reduce system power consumption and improve deployment flexibility. However, existing RIS typically only reflect signals, meaning they can only serve users located on the same side of the network. This unidirectional signal processing method makes it difficult to meet the communication needs of users on the other side, resulting in limitations in spatial coverage and deployment flexibility.

[0025] Based on this, the embodiments of this application achieve simultaneous transmission and reflection control of incident signals by introducing the synergistic effect of shared power distribution circuits and shared phase shifting circuits in the sub-arrays. This overcomes the spatial limitation of traditional unidirectional reflective surfaces that can only serve users on the same side, effectively expanding the spatial coverage of the communication system and improving deployment flexibility. Furthermore, by adopting a sub-array architecture, multiple reflection and transmission units share amplification, power distribution, and phase shifting circuits, significantly reducing the number of hardware components and the static power consumption of the basic circuits. At the same time, combined with a mode switching switch, each sub-array can flexibly switch between different operating modes based on the received control signals. Thus, in self-powered scenarios that rely on environmental energy harvesting, the system can perform finer-grained scheduling and efficient utilization of limited energy resources according to the dynamically changing battery energy state. This effectively alleviates the problem of shortened effective working time caused by insufficient or fluctuating environmental energy supply, thereby significantly improving the long-term average throughput and communication reliability of the communication system while reducing overall system power consumption.

[0026] The following will further describe the simultaneous transmission reflector subarray reflector, the parameter optimization method for the communication system, and related equipment proposed in the embodiments of this application. First, a simultaneous transmission reflector subarray reflector will be described, referring to… Figure 1 This is a schematic diagram of the structure of a simultaneous transmission and reflection subarray reflector provided in an embodiment of this application; see reference. Figure 2 This is a schematic diagram of another simultaneous transmission and reflection subarray reflective surface provided in an embodiment of this application. For example... Figure 1 and Figure 2As shown, the reflective surface of the simultaneous transmission and reflection subarray includes an energy harvesting circuit, a battery cell, and multiple subarrays.

[0027] In one embodiment, the energy harvesting circuit is used to harvest ambient energy and store it in battery cells to power multiple subarrays. Specifically, the energy harvesting circuit is used to obtain energy from the surrounding environment (such as solar energy, wind energy, etc.). The reflective surface of the simultaneous transmission and reflection subarray can simultaneously reflect, transmit signals, and harvest energy in each time slot. The energy harvesting circuit stores the ambient energy collected in each time slot into the battery cells, and then consumes the energy stored in the battery cells when the reflective surface of the simultaneous transmission and reflection subarray is running to maintain the operation of each subarray.

[0028] Each subarray consists of G reflective-transmitting units, and each subarray includes a mode switching switch, a shared phase shifting circuit, a shared power distribution circuit, a shared amplification circuit, and multiple reflective-transmitting units. Specifically, the reflective surface of the simultaneous transmission and reflection subarray is divided into several subarrays (i.e., L subarrays), and each subarray consists of multiple reflective-transmitting units (i.e., ... The array consists of N reflective and transmissive units, where N is the total number of reflective and transmissive units on the reflective surface of the reflective subarray. In each subarray, all reflective and transmissive units share a set of circuits, specifically including an amplifier (corresponding to a shared amplifier circuit), a power divider (corresponding to a shared power distribution circuit), and a reflection phase shift circuit and a transmission phase shift circuit (together corresponding to a shared phase shift circuit).

[0029] When the reflective surface of the simultaneous transmission and reflection subarray receives a mode control signal, the corresponding mode switching switch is controlled based on this signal to control the corresponding reflection and transmission unit to switch between different operating modes (idle mode, passive mode, and active mode). Specifically, combined with Figure 2 As shown, each subarray is configured with a single-pole double-throw switch (corresponding to...). Figure 2 The nodes b, a, and c in the array are used to control the mode switching of the subarray, thereby changing the communication links for transmission modulation and reflection modulation.

[0030] In idle mode, the mode switch is off. Specifically, as follows: Figure 2 As shown, when the switch is in the off state, the subarray enters an idle mode, and all units in the subarray will not reflect or transmit signals.

[0031] In passive mode, the mode switch directly connects the input incident signal to the shared power distribution circuit. Specifically, as shown below... Figure 2As shown, when the switch is closed downwards (i.e., node b is connected to node c), the amplifier is bypassed, and the components in the subarray switch to passive mode. At this time, the incident signal from the reflection or transmission region is split into two paths after passing through the power divider, and enters the phase shift control circuits for reflection and transmission respectively for phase modulation.

[0032] In active mode, the mode switch first amplifies the input incident signal through a shared amplifier circuit before it is input to the shared power distribution circuit. Specifically, as follows... Figure 2 As shown, when the switch is closed upwards (i.e., node b is connected to node a), all elements in the subarray enter active mode. At this time, the incident signal is first amplified by an amplifier, and then split into two paths by a power divider, which enter the phase shift control circuits for reflection and transmission respectively, so as to output the reflected signal and the transmitted signal simultaneously.

[0033] In one example, let A vector representing the switching states of all subarrays, where Indicates the first ( The switching states of each subarray. When Time indicates the first When the switches corresponding to each subarray are in the off state, all units within the subarray enter an idle mode and will not reflect or transmit signals. Time indicates the first The switches corresponding to each subarray close downwards, bypassing the amplifier and switching the components within the subarray to passive mode. At this point, the incident signal, after passing through the power divider, is split into two paths, which enter the phase shift control circuits for reflection and transmission, respectively, for phase modulation. When Time indicates the first When the switch corresponding to each subarray is closed upwards, all components within that subarray enter active mode. At this time, the incident signal is first amplified by an amplifier, and then split into two paths by a power divider, which enter the phase shift control circuits for reflection and transmission respectively.

[0034] In this application, the battery energy consumption of the simultaneous transmission-reflection subarray reflector is assumed to be as follows: the simultaneous transmission-reflection subarray reflector can simultaneously reflect (transmit) signals and harvest energy in each time slot. The simultaneous transmission-reflection subarray reflector STAR-RIS stores the energy harvested in each time slot into the battery; simultaneously, STAR-RIS consumes the energy stored in the battery to assist the user in uploading information to the base station. It is particularly noteworthy that... Time slots, STAR-RIS can only use Time slots and previously stored energy; while The energy collected in a new time slot, as well as any remaining energy from previous time slots, is carried over to subsequent time slots.

[0035] The simultaneous transmission and reflection subarray reflector provided in this application divides the reflector into several subarrays, allowing all reflection and transmission units in each subarray to share amplifiers, power dividers, and phase-shifting circuits. This not only overcomes the spatial limitations of traditional unidirectional reflectors in serving users on the other side, achieving full-space signal modulation, but also effectively reduces hardware costs and static circuit power consumption caused by configuring independent circuits for each component. Furthermore, through a mode switching mechanism controlled by a single-pole double-throw switch, each subarray can switch between active, passive, and idle modes based on control signals. This allows for more granular energy resource scheduling based on the dynamic state of available battery energy in self-powered communication scenarios that rely on environmental energy harvesting, effectively improving the long-term average throughput of the system while meeting energy constraints.

[0036] Reference Figure 3 This application provides a schematic diagram of the structure of a communication system. The communication system mainly includes a base station (equipped with M antennas) located in a reflection area. Multiple first user terminals (i.e. Figure 3 (The user is in the reflection area) and located in the transmission area. Multiple second user terminals (i.e. Figure 3 The transmission area of ​​the user) and the simultaneous transmission and reflection subarray reflector deployed between the base station and the user (i.e., the reflector surface of the transmission and reflection subarray) Figure 3 The STAR-RIS, a self-powered system based on a hybrid subarray architecture and equipped with N reflection and transmission units, consists of the following components: ), where the total number of the first user terminal and the second user terminal is J, and the spatial region direction set and the user set are respectively and .

[0037] This application will The user set of the area is set as , The user set of the area is set as ,satisfy For ease of representation, use Representative at The first direction The proposed simultaneous transmission and reflection subarray can simultaneously reflect and transmit signals to multiple users.

[0038] In this communication system, the direct communication links between the base station and each user terminal are usually obstructed by physical barriers (such as...). Figure 3The simultaneous transmission and reflection subarray (as shown in the image) blocks the signal, while the reflective surface of the simultaneous transmission and reflection subarray is used to assist in establishing a reliable communication connection. Specifically, the reflective surface of the simultaneous transmission and reflection subarray is equipped with a controller and an energy harvesting circuit. The energy harvesting circuit is used to collect external environmental energy (such as solar energy, wind energy, etc.) and convert it into electrical energy for storage to maintain the operation of the array itself. During operation, under the control of the controller, different subarrays within the simultaneous transmission and reflection subarray can operate in active mode, passive mode, or idle mode, respectively. This allows it to receive uplink transmission signals from the second user terminal in the transmission area and the first user terminal in the reflection area. By utilizing the array structure for coordinated control of transmission and reflection, the processed signals are finally transmitted to the base station, thereby achieving full-space communication coverage and auxiliary transmission for users located on both sides of the reflective surface.

[0039] In the communication system of this application, the channel model is defined as follows: It is assumed that all channels are quasi-static, meaning the channels remain unchanged within each time slot. The user is defined. The direct link channel coefficient to the base station is (Including the first channel state and the second channel state), user The channel coefficient to the STAR-RIS link is (i.e., the fourth channel state), the channel coefficient matrix of the STAR-RIS to base station link is as follows: (i.e., the third channel state). The reflection matrix of STAR-RIS in the reflection direction is defined as shown in the following formula: ; in, ; Indicates the first Power allocation coefficients of each subarray Indicates the first The amplitude coefficient of each subarray Indicates the first Phase shift of reflections in each subarray.

[0040] The transmission matrix of STAR-RIS in the transmission direction is defined by the following formula: ; in, ; Indicates the first Transmission phase shift of each subarray.

[0041] Based on the above definition, the base station in The signal received in the time slot can be expressed as shown in the following formula: ; in, and These are user terminals (Including the first user terminal and the second user terminal) in The transmission power and signal of the time slot; The noise amplification factor matrix of the STAR-RIS active components in the direction is shown in the following formula: ; in, ; The active elements in the hybrid-subarray STAR-RIS are introduced to have a mean of 0 and a variance of 0. Thermal noise with a complex Gaussian distribution; The noise at the base station is additive white Gaussian noise with a mean of 0 and a variance of 1. The complex Gaussian distribution.

[0042] To decode from users The signal, the base station uses Receiver beamformer The mathematical expression for the received user signals is: .

[0043] Based on the above description of the simultaneous transmission and reflection subarray reflector and communication system, the following will further describe a method for optimizing transmission parameters of a communication system provided in an embodiment of this application. The method for optimizing transmission parameters of a communication system provided in this embodiment can be applied to a base station in a communication system or a processor connected to a base station, etc. (Refer to...) Figure 4 This is an optional flowchart of a method for optimizing transmission parameters of a communication system provided in an embodiment of this application. Figure 4 The method described may include, but is not limited to, steps 100 to 500. It is also understood that this embodiment... Figure 4 The order of steps 100 to 500 is not specifically limited. The order of steps can be adjusted or some steps can be reduced or added according to actual needs.

[0044] Step 100: Obtain the first channel state information between each first user terminal and the base station, the second channel state information between each second user terminal and the base station, the third channel state information between the simultaneous transmission and reflection subarray reflector and the base station, and the fourth channel state information between each first user terminal and the second user terminal and the simultaneous transmission and reflection subarray reflector, and obtain the battery energy state information of the simultaneous transmission and reflection subarray reflector in the current time frame.

[0045] Step 100 is described in detail below.

[0046] In step 100 of some embodiments, the system first needs to acquire basic state data of the current communication environment as a basis for subsequent parameter optimization decisions. The base station, acting as an agent, observes the quasi-static channel state coefficients between the aforementioned communication nodes in the current time slot system, as well as the initial available energy of the reflector in the current time slot, and combines these to construct the current state variables of the Markov decision process, thereby providing an environmental awareness basis for the subsequent decisions of the network model; wherein, the first channel state information... This refers to the channel fading parameters of the direct link from the first user terminal to the base station in the reflection area, and the second channel state information. This refers to the channel fading parameters of the direct link from the second user terminal to the base station in the transmission area, and the third channel state information. This characterizes the channel coefficient matrix of the link between the simultaneous transmission and reflection subarray reflector and the base station, and the fourth channel state information. Channel coefficients and battery state information characterizing the links from each user terminal to the reflector surface of the simultaneous transmission and reflection subarray. This refers to the available energy stored in the battery cell at the initial moment of the current time slot by the reflective surface of the simultaneous transmission and reflection subarray.

[0047] Step 200: Input the first channel state information, the second channel state information, the third channel state information, the fourth channel state information, and the battery energy state information into the transmission parameter optimization model for data processing to obtain optimization parameters. The optimization parameters include the optimized mode switching sequence, the optimized power allocation coefficient vector sequence, the optimized reflection phase shift sequence, the optimized transmission phase shift sequence, the optimized amplification coefficient sequence, the optimized reflection transmission power, and the optimized transmission transmission power.

[0048] Step 200 is described in detail below.

[0049] In step 200 of some embodiments, the acquired comprehensive state information is input into the transmission parameter optimization model to sample and output joint discrete actions and joint continuous actions through the model's policy distribution. Through data inference processing of the model, the system can quickly obtain joint optimization parameters in a complex high-dimensional hybrid action space, providing specific variable instructions for the subsequent coordinated control of the communication system. The transmission parameter optimization model is a network model pre-trained through deep reinforcement learning to output a joint resource optimization strategy. The optimization parameters include an optimization mode switching sequence. Optimize the power allocation coefficient vector sequence Optimize the reflection phase shift sequence Optimize transmission phase shift sequence Optimize the amplification factor sequence Optimize reflected transmission power and optimize transmission power Among them, the optimized mode switch sequence is a set of command variables used to indicate the open or closed state of each subarray switch inside the reflector; the optimized power allocation coefficient vector sequence is a continuous variable used to indicate the power division ratio of transmitted and reflected signals within the subarray; the optimized reflection phase shift sequence and the optimized transmission phase shift sequence are sets of discrete quantized values ​​used to regulate the phase of reflected and transmitted signals, respectively; the optimized amplification coefficient sequence is a continuous variable used to control the amplification degree of the active element on the signal; and the optimized reflection transmission power and the optimized transmission transmission power are signal transmission power parameters allocated to each user terminal in the reflection and transmission regions, respectively.

[0050] The construction steps of the transmission parameter optimization model include steps 210 to 240.

[0051] Step 210: Based on the reflection phase shift parameter sequence, transmission phase shift parameter sequence, amplification factor parameter sequence, power allocation factor parameter sequence, mode switching parameter sequence of the reflective surface of the simultaneous transmission reflector subarray, as well as the transmission power parameters of each first user terminal and second user terminal and the beamforming parameters of the base station, construct the first decoding signal-to-noise ratio function of the reflection region and the second decoding signal-to-noise ratio function of the transmission region, respectively.

[0052] Step 220: Based on the first decoding signal-to-noise ratio function and the second decoding signal-to-noise ratio function, obtain the average throughput objective function, and construct the parameter optimization problem based on the average throughput objective function.

[0053] Steps 210 to 220 are described in detail below.

[0054] In step 210 of some embodiments, it is first necessary to mathematically model the signal quality during the communication process. Therefore, it is necessary to comprehensively consider the fading characteristics and thermal noise of each user terminal to the reflector, the reflector to the base station, and the user to the base station in advance, and combine the above-mentioned multi-dimensional underlying control variables to construct the first decoding signal-to-noise ratio function and the second decoding signal-to-noise ratio function for the reflection direction and the transmission direction, respectively. Among them, the reflection phase shift parameter sequence and transmission phase shift parameter sequence These are discrete sets of coefficients used to characterize the phase adjustment of the reflected and transmitted signals by the subarrays, respectively, and are amplification coefficient parameter sequences. These are variables used to characterize the signal power gain of the internal amplifiers of the subarray in active mode; they are power allocation coefficient parameter sequences. These are parameter variables used to characterize the proportion of energy of the incident signal distributed to the reflecting and transmitting sides; mode switching parameter sequence. These are single-pole double-throw switch state variables used to indicate whether each subarray is operating in passive, active, or idle mode, and transmit power parameters (including...). and Beamforming parameters are used to characterize the transmit power used by each user terminal when sending uplink signals. The base station uses receive beamforming matrix parameters for receiving and equalizing multi-user signals. The first decoding signal-to-noise ratio function is a mathematical expression that characterizes the ratio of effective signal power to total noise power when the base station decodes the signal transmitted by the first user terminal in the reflection area. The second decoding signal-to-noise ratio function is a mathematical expression that characterizes the ratio of effective signal power to total noise power when the base station decodes the signal transmitted by the second user terminal in the transmission area.

[0055] In this application, The first time slot reflection direction The first signal-to-interference-plus-noise ratio (SIR) function of the user's transmitted signal decoded at the base station is shown in the following formula: ; in, .

[0056] Similarly, in The first time slot transmission direction The second signal-to-interference-plus-noise ratio (SIR) function of the user's transmitted signal decoded at the base station is shown in the following formula: ; In step 220 of some embodiments, after establishing the underlying signal-to-noise ratio (SNR) model, the system calculates the transmission rate that each user can achieve in the corresponding time slot based on Shannon's theorem, combined with the first and second decoding SNR functions, and then accumulates the data to obtain the average throughput objective function of the system in multiple time slots, as shown in the following formula:

[0057] The average throughput objective function is a mathematical function used to measure the average data transmission rate that a communication system can support over a relatively long period of time.

[0058] Then, guided by the objective function of maximizing the average throughput, and taking into account conditions such as the minimum data volume constraints for each user and the maximum transmit power constraints for the device, a mixed integer nonlinear programming (MINLP) parameter optimization problem with deep coupling of discrete and continuous variables is constructed, as described in detail.

[0059] The parameter optimization problem is constructed based on the average throughput objective function, including the following steps 221 to 225.

[0060] Step 221: Based on the static power consumption of the reflection phase shift control circuit, the static power consumption of the transmission phase shift control circuit, the static power consumption of the power divider, the static power consumption of the amplifier in the shared amplifier circuit, and the activation state of the operating mode of each subarray indicated by the mode switch parameter sequence, the total static circuit power consumption of the reflective surface of the simultaneous transmission and reflection subarray is obtained.

[0061] Step 222: Based on the amplifier conversion efficiency, fourth channel state information, transmit power parameters, power allocation coefficient parameter sequence, amplification coefficient parameter sequence, and input noise variance, obtain the dynamic amplification power consumption of the reflective surface of the simultaneous transmission reflector subarray.

[0062] Step 223: Based on the sum of the total static circuit power consumption and the dynamic amplification power consumption, obtain the system energy consumption item for the current time frame.

[0063] Step 224: Based on the battery energy state information of the previous time frame, the environmental energy acquisition items of the current time frame, the system energy consumption items, and the maximum battery capacity, perform boundary constraint calculations to obtain the battery energy update function.

[0064] Step 225: Construct the parameter optimization problem based on the average throughput objective function and the battery energy update function.

[0065] Steps 221 to 225 are described in detail below.

[0066] In step 221 of some embodiments, it is necessary to perform accurate mathematical modeling of the basic hardware power consumption of the reflector. The number of subarrays currently in active and passive modes is identified by the mode switch parameter sequence. The fixed power consumption of the reflection phase shift circuit, transmission phase shift circuit, power divider, and amplifier in the working state is then superimposed and calculated to obtain the total static circuit power consumption within the time frame, as shown in the following formula: ; in, and These represent the power consumption of each phase shift control circuit and power amplifier, respectively. It is an indicator function, when The function value is 1 when the condition is met, and 0 otherwise. The duration of each time slot, The power consumption of the passive component's power divider is shown in the following formula: ; in, ; It is the insertion power loss factor of the power divider.

[0067] In step 222 of some embodiments, after determining the static power consumption, it is also necessary to calculate the variable energy consumption of the components in active mode. The actual incident signal strength reaching the reflector is calculated based on the fourth channel state information and the transmit power parameters of each terminal. Then, mathematical operations are performed by combining the required power allocation coefficient parameter sequence, amplification coefficient parameter sequence, and the input noise variance and amplifier conversion efficiency introduced by the components themselves, so as to calculate the dynamic amplification power consumption caused by the power amplification of the signal by all subarrays in active mode, as shown in the following formula: ; Dynamic amplification power consumption is used to quantify the variable power consumption of a subarray in active mode to amplify the incident signal to the desired intensity. The power consumption of the power divider for active components is shown in the following formula: ; in, ; In step 223 of some embodiments, it is necessary to integrate power consumption data from different dimensions to form a complete energy consumption metric. The accumulated value refers to the total energy data obtained by numerically summing the fixed static energy consumption of the hardware module and the dynamic variable energy consumption of the signal amplification, i.e. The system energy consumption term is used to quantify the total electrical energy consumed by the entire simultaneous transmission and reflection subarray reflector to complete the transmission and reflection coordinated auxiliary task within a single time frame period.

[0068] In step 224 of some embodiments, in order to ensure that the energy evolution state across time slots conforms to the actual physical conservation law, the present application scheme extracts the remaining energy of the battery at the end of the previous time frame. Subtract the system energy consumption item for the current time frame. In addition, the environmental energy harvesting item obtained from the environment in the current time frame is added. (its obedience) (uniform distribution); then, the calculation results are compared with the battery's maximum capacity. By performing boundary constraint calculations and comparing the two values, and taking the smaller value, the battery energy update function is finally constructed as shown in the following formula: ; The battery energy update function is an equation expression used to describe the evolution of the remaining available energy inside the battery between adjacent time frames and the recursive relationship across time slots.

[0069] In step 225 of some embodiments, the average throughput objective function used to evaluate long-term communication performance is used as the optimization solution objective of the planning model, and the battery energy update function is incorporated into the model as a core constraint, thereby constructing a complete parameter optimization problem that fits the actual energy harvesting characteristics, as described below.

[0070] The parameter optimization problem, which is constructed based on the average throughput objective function and the battery energy update function, includes the following steps 226 to 229.

[0071] Step 226: Obtain the current available energy of the battery, and construct energy causal constraints based on the numerical relationship between the system energy consumption item and the current available energy of the battery. The energy causal constraints indicate that the energy consumed in the current time frame does not exceed the stored energy of the battery at the beginning of the time frame.

[0072] Step 227: Based on the first decoding signal-to-noise ratio function and the second decoding signal-to-noise ratio function, calculate the total amount of data uploaded by a single user in the reflection area and the transmission area respectively, and construct service quality constraints based on the numerical relationship between the total amount of data uploaded by a single user and the minimum service quality threshold.

[0073] Step 228: Based on the numerical relationship between the transmit power parameter and the maximum transmit power threshold, the numerical relationship between the amplification factor parameter sequence and the maximum amplification threshold, the numerical relationship between the power allocation factor parameter sequence and the power allocation factor range, and the matching relationship between the mode switching parameter sequence, the reflection phase shift parameter sequence, the transmission phase shift parameter sequence and the discrete operating space, construct the physical constraints of the device.

[0074] Step 229: Based on the average throughput objective function, battery energy update function, energy causal constraints, service quality constraints, and equipment physical constraints, a parameter optimization problem is constructed.

[0075] Steps 226 to 229 are described in detail below.

[0076] In step 226 of some embodiments, before parameter optimization, it is necessary to ensure that the energy consumption of the hardware device conforms to objective physical laws. The solution of this application obtains the current available battery energy and compares and restricts it with the system energy consumption item calculated in the aforementioned steps, thereby constructing corresponding energy causal constraints in the model. This is a boundary constraint expression that limits the energy expenditure of the system in any time slot to the sum of its historical accumulation and current holdings.

[0077] In step 227 of some embodiments, in addition to energy considerations, it is also necessary to ensure the basic communication needs of each communication terminal. Therefore, the solution of this application, based on the first decoding signal-to-noise ratio function and the second decoding signal-to-noise ratio function, combined with the time allocation characteristics of the system, calculates the total amount of data uploaded by each user in the reflection area and the transmission area, respectively. Then, it establishes a numerical comparison relationship between the total amount of data uploaded by each user and a preset minimum service quality threshold, thereby constructing a service quality constraint condition. This is to ensure that, in the process of pursuing an increase in the overall throughput of the system, the basic communication rights of individual users are taken into account without being compromised.

[0078] In step 228 of some embodiments, the selection of system control parameters is limited by the actual performance indicators of the hardware device. In this application, numerical constraints are established between the transmit power parameter and the maximum transmit power threshold, the amplification factor parameter sequence and the maximum amplification threshold, and the power allocation factor parameter sequence and the power allocation factor range. Simultaneously, numerical matching relationships are also established between the mode switching parameter sequence, the reflection phase shift parameter sequence, and the transmission phase shift parameter sequence and the discrete operating space (such as a finite set of phase shift quantization bits). , , In this way, the physical constraints of the device are summarized and constructed to prevent the model from outputting erroneous control commands that exceed the execution capabilities of the underlying hardware.

[0079] In step 229 of some embodiments, the communication objectives of each dimension are finally mathematically integrated with the underlying constraints. The average throughput objective function, which reflects long-term transmission performance, is used as the optimization objective, and the battery energy update function is used as the key constraint for cross-time slot state transition. Combined with the constructed energy causal constraints, service quality constraints, and device physical constraints, the parameter optimization problem is constructed as shown in the following formula (P1): (a) (b) (c) (d) (e) (f) (g) (h) (i) (j) Wherein, equation (b) represents STAR-RIS in Battery energy in time slots and The relationship between battery energy, energy consumption, and harvested energy in a time slot; Equation (c) represents the indicator for the closure of the single-pole double-throw switch that controls the operating mode of each subarray; Equation (d) limits the user's transmit power. This is the maximum transmission power for each user; Equation (e) is a constraint on the STAR-RIS amplification factor, where It is the maximum amplification factor of the active reflective (transmittive) element; Equation (f) is the constraint on the minimum amount of data uploaded by a single user in each time slot; Equation (g) is the discrete phase shift constraint of STAR-RIS, assuming that the phase shift of the reflective unit of STAR-RIS is quantized in D bits, thus obtaining the set of discrete phase shift values; Equation (h) shows that STAR-RIS in The energy that can be consumed in a time slot cannot exceed the energy stored in the battery at the beginning of that time slot; Equation (i) is a constraint on the STAR-RIS power allocation coefficient; Equation (j) is a normalization constraint on the beamforming of the received signal at the base station.

[0080] Through steps 221 to 229 above, the system power consumption of the reflector is refined into total static circuit power consumption and dynamic amplification power consumption. A battery energy update function is constructed based on the actual environmental energy harvesting conditions and the upper limit of battery capacity. This allows the optimization model to accurately reflect the real energy flow and consumption mechanism within the self-powered device. Furthermore, by introducing energy causal constraints, quality of service constraints, and equipment physical constraints during the parameter optimization problem construction process, the final mathematical model can realistically map the physical limitations and service requirements of the simultaneous transmission and reflection subarray reflector in the actual operation under a self-powered environment. This effectively avoids the optimization algorithm falling into a control blind spot detached from physical reality during the solution exploration, ensuring that the control strategy output by the subsequently trained transmission parameter optimization model not only significantly improves the overall average throughput of the system but also possesses high executability and system operational stability during engineering implementation.

[0081] Due to binary optimization variables With continuous optimization variables , The deep coupling between variables, and the cross-timeslot use of STAR-RIS battery energy, further enhances the temporal coupling between variables, making the solution space of the problem highly complex. Problem (P1) is also a challenging DP and MINLP problem. This section will introduce a hierarchical DRL optimization method to solve problem (P1).

[0082] Step 230: Based on the beamforming parameters, the parameter optimization problem is divided into a lower-level beamforming optimization sub-problem and an upper-level resource joint optimization sub-problem.

[0083] Step 230 will be described in detail below.

[0084] In step 230 of some embodiments, the binary indicator variable and the continuous adjustment variable are deeply coupled in the constructed optimization problem, and there is energy update across time slots, making direct solution too complex. Therefore, the system performs dimensionality reduction decomposition on this complex problem. The solution in this application adopts a hierarchical optimization structure, decomposing the originally highly coupled original parameter optimization problem into the two logically distinct sub-problems mentioned above. This effectively reduces the difficulty of solving the solution space, enabling the system to use appropriate algorithmic routes to process beamforming parameters with closed solutions and resource allocation parameters lacking closed solutions, respectively.

[0085] In this application, to solve problem (P1) using a DRL-based hierarchical optimization method, problem (P1) is first decomposed into a lower-level beamforming optimization sub-problem and an upper-level resource joint optimization sub-problem. The upper-level resource joint optimization sub-problem is based on optimal base station receive beamforming, and the optimization variables are solved using the DRL method. It can be expressed as the following formula (P1.1): ; ; Problem (P1.1) remains a complex DP and MINLP problem. The proposed solution will adopt a DRL algorithm based on PPO, and learn the optimal policy by constructing and solving an MDP problem to obtain the optimal solution to problem (P1.1).

[0086] The lower-level beamforming optimization sub-problem is based on a given... The optimal base station receiving beamforming vector can be obtained by solving the following formula (P1.2): ; Using a linear MMSE detector, the optimal linear beamforming for receiving user signals in the reflected and transmitted regions at the base station is solved. Its analytical expression can be represented as follows: ; ; The final optimized reflection receiving beamforming and optimized transmission receiving beamforming are obtained by the following formulas.

[0087] , ; Step 240: Construct an initial transmission parameter optimization model based on the upper-layer resource joint optimization sub-problem, and perform multiple rounds of training on the initial transmission parameter optimization model to obtain the transmission parameter optimization model.

[0088] Step 240 will be described in detail below.

[0089] In step 240 of some embodiments, deep reinforcement learning techniques are introduced to establish an intelligent learning mechanism for the upper-layer resource optimization subproblem (P1.1) which lacks a closed-form solution. The proposed solution first reformulates the upper-layer resource joint optimization subproblem (P1.1) as a Markov decision process, establishing the state space, action space, and reward signal of this process, thereby constructing an initial transmission parameter optimization model containing discrete and continuous action output structures. Subsequently, using the base station as the agent, actions are continuously taken based on the current strategy in a simulation environment. Multiple rounds of network parameter training and gradient updates are performed by collecting experienced channel environment and energy state transition data until the algorithm converges and stabilizes, ultimately obtaining a transmission parameter optimization model that can adapt to dynamic wireless environments.

[0090] The following section first describes how to construct the initial transmission parameter optimization model.

[0091] The initial transmission parameter optimization model is constructed based on the joint optimization sub-problem of upper-layer resources, including the following steps 241 to 245.

[0092] Step 241: Based on the first channel state information, the second channel state information, the third channel state information, the fourth channel state information, and the available energy state of the battery, obtain the state space.

[0093] Step 242: Obtain the continuous action space based on the amplification factor parameter sequence, the power allocation factor parameter sequence, and the transmission power parameter; obtain the discrete action space based on the mode switching parameter sequence, the reflection phase shift parameter sequence, and the transmission phase shift parameter sequence.

[0094] Step 243: Based on the average throughput objective function in the upper-level resource joint optimization subproblem, construct the throughput reward function.

[0095] Step 244: Obtain the initial parameters of the critic network, the initial parameters of the discrete actor network, and the initial parameters of the continuous actor network that outputs the joint continuous action.

[0096] Step 245: Based on the state space, continuous action space, discrete action space, throughput reward function, initial commentator network parameters, initial discrete actor network parameters, and initial continuous actor network parameters, obtain the initial transmission parameter optimization model.

[0097] Steps 241 to 245 are described in detail below.

[0098] In step 241 of some embodiments, firstly, it is necessary to convert the feature information of the communication environment into environmental input variables that can be recognized by a deep reinforcement learning algorithm, due to the channel state coefficients of the aforementioned links and the available battery energy of the reflector in the current time slot. This directly affects the subsequent selection of parameters such as the subarray operating mode and amplification factor by the system. Therefore, the system observes this real-time information to construct the state space of a Markov decision process, which serves as the basic input for the transmission parameter optimization model to perceive the communication environment. The state space is shown in the following formula: ; Wherein, the dimension of the state space is .

[0099] In step 242 of some embodiments, it is necessary to clarify the boundaries of the operation instructions that the agent can output when making decisions. Since the upper-layer resource joint optimization subproblem contains a large number of variables related to the number of subarrays, direct optimization will lead to excessive dimensionality. Therefore, the solution of this application adopts the action combination method, which combines the above-mentioned independent actions such as amplification coefficient, power allocation coefficient and transmission power into a joint continuous action space, and combines the independent actions such as mode switching, reflection phase shift and transmission phase shift into a joint discrete action space, so as to obtain the overall action space as shown in the following formula: ; Among them, the dimension of the action space is .

[0100] In this application's scheme, among the optimization variables of the upper-level resource joint optimization sub-problem (P1.1), It is a binary variable, and They are also discrete variables; other optimization variables These are continuous variables. The action space of the constructed MDP is still a mixed discrete-continuous action space. The proposed solution introduces the HPPO algorithm based on action combination to solve the problem (P1.1).

[0101] Because the optimization variables in problem (P1.1) include The number of actions is equal to the number of subarrays. Related. Due to Typically, the size is quite large. If all optimization variables are treated as independent actions, it will lead to a surge in the dimensionality of the action space (the discrete action space has a dimensionality of 100 dimensionality). The spatial dimension of continuous actions is ).

[0102] The action space for the MDP constructed in this application is described in detail below. First, the MDP is... Each time slot action is divided into joint discrete actions and joint continuous actions, using a discrete action space. and continuous action space express.

[0103] In the HPPO algorithm based on action combination, the policy network directly outputs the probability distribution of each independent sub-action. Therefore, the probability distribution of the joint discrete actions is expressed as follows: ; in, These represent the states respectively. The policy involves determining the on / off states of all subarrays and selecting the reflection (transmission) phase shift. The policy network output value for each independent sub-action has an unnormalized logarithmic probability, which is then normalized using a softmax function to obtain the probability distribution for each independent sub-action. Definition, the first Switches of subarrays There are three possible values: phase shift. Possible values ​​are Therefore, the number of neurons in the output layer of the discrete actor network is . If the action combination method is not applied, the number of neurons in the output layer of the discrete actor network is [number missing]. This demonstrates that the action combination method can effectively address the problem of high-dimensional discrete actions.

[0104] The probability distribution of joint continuous actions is expressed by the following formula: ; in, These represent the states respectively. The strategy for selecting the power allocation coefficients, amplitude coefficients, and transmit power actions of users in the reflection (transmission) region for all subarrays is described. Each independent continuous sub-action is obtained through Beta sampling, and the sampled values ​​are transformed to the actual constraint interval through linear transformation.

[0105] In step 243 of some embodiments, the system needs to provide quantitative directional guidance for the policy exploration process of the reinforcement learning algorithm. The solution of this application transforms the optimization orientation aimed at maximizing long-term throughput in the upper-layer resource joint optimization sub-problem into the instant reward feedback obtained by the agent after interacting with the environment in each time slot. That is, the corresponding throughput reward function is directly constructed based on the average throughput objective function, thereby driving the underlying algorithm to continuously try and learn towards the optimal policy direction of improving the system transmission rate. The throughput reward function is shown in the following formula: ; In step 244 of some embodiments, it is necessary to initialize the internal structure of the deep neural network used to implement the Hybrid Proximal Policy Optimization (HPPO) algorithm. Therefore, this is done by acquiring and setting the initial network parameters (including the initial critic network parameters) without loaded empirical data. Initial discrete actor network parameters And the initial continuous actor network parameters of the continuous actor network that outputs joint continuous actions. This paper establishes a basic framework for a hybrid network consisting of a dual-parallel actor network and a single-shared critic network, laying the physical computational foundation for subsequent parameter optimization and gradient updates in a discrete-continuous hybrid action space. Specifically, the critic network is used to evaluate the long-term profit value prediction network that the system can obtain by following a specific strategy in the current state; the discrete actor network is a decision network specifically designed to handle high-dimensional discrete operation spaces and output the corresponding discrete action probability distributions; and the continuous actor network is a decision network specifically designed to handle continuous operation spaces and output the corresponding continuous action value strategies.

[0106] In step 245 of some embodiments, the established state space is configured as the perceptual input layer of the network, and the continuous action space and discrete action space are configured as the action output layers of the corresponding actor networks. The throughput reward function is used as the feedback criterion for the critic network to evaluate the quality of the strategy. The initial critic network parameters, initial discrete actor network parameters, and initial continuous actor network parameters are loaded internally. In this way, an initial transmission parameter optimization model that can cope with the solution of complex communication links is integrated and constructed, thus preparing the basic computing carrier for the subsequent multi-round environmental interaction simulation and model weight training.

[0107] Through steps 241 to 245 above, the complex upper-layer resource joint optimization subproblem is mapped to a standard Markov decision process. Multi-dimensional channel fading information and battery energy reserves are constructed into a unified state space. The action combination method is used to reasonably separate and dimensionality-reduced combine discrete and continuous operational variables, thereby constructing an initial transmission parameter optimization model. This effectively overcomes the technical bottlenecks of traditional reinforcement learning in handling mixed action spaces and the surge in action space dimensions caused by the quantum array. It lays the architectural foundation for stable and efficient training and updating of subsequent models, enabling the system to more flexibly cope with time-varying self-powered communication environments.

[0108] The initial transmission parameter optimization model is trained through multiple rounds of reinforcement learning, including steps 246 to 2410.

[0109] Step 246: In each training time step, the current state information is input into the discrete actor network and the continuous actor network respectively, and the current discrete action and the current continuous action are obtained by sampling through the policy distribution respectively.

[0110] Step 247: Solve the lower-level beamforming optimization subproblem based on the current discrete action and the current continuous action to obtain the current receiving beamforming vector.

[0111] Step 248: Apply the current discrete action, the current continuous action, and the current received beamforming vector to the system training environment to obtain the current throughput reward calculated based on the throughput reward function, and obtain the updated state information for the next time step.

[0112] Step 249: Based on the current state information, current discrete action, current continuous action, current throughput reward and update state information, obtain the record information and store the record information in the experience replay pool.

[0113] Step 2410: When the preset update period is met, a small batch of samples is extracted from the experience replay pool. Based on the advantage function, pruning and substitution objective function and policy entropy calculated by generalized advantage estimation, gradient updates are performed on the initial commentator network parameters, initial discrete actor network parameters and initial continuous actor network parameters, respectively.

[0114] Steps 246 to 2410 are described in detail below.

[0115] In step 246 of some embodiments, the system enters a trial-and-error phase of reinforcement learning in a simulated environment. In each training step, the base station, acting as the agent, first observes the current channel and energy parameters. and the current status information The inputs are fed in parallel into the two independent actor networks mentioned above; the networks calculate the probability distribution of the output action, i.e., the current discrete action strategy, through forward propagation. and continuous action strategy The system then samples based on these distributions to generate the current discrete action. and current continuous actions ,action and Combined into a complete action in MDP This provides preliminary decision variables for subsequent closed-loop calculations.

[0116] In step 247 of some embodiments, after obtaining the current discrete action... and current continuous actions Then, it is substituted as a known condition into the lower-level beamforming optimization subproblem (P1.2). By using mathematical rules such as the linear minimum mean square error detector, a closed-form analytical solution is performed to quickly obtain the current receiving beamforming vector (i.e., optimizing the reflection receiving beamforming and optimizing the transmission receiving beamforming). Thus, the reasoning calculation of all parameters to be optimized in the entire communication cascade link at the current time step is completed.

[0117] In step 248 of some embodiments, all control action variables generated by the above combination are sent to the virtual environment to simulate the real user transmission and base station decoding process in the system training environment, and the current system transmission sum is calculated as the current throughput reward based on the throughput reward function. Subsequently, the system deduces a new battery state based on system energy consumption and environmental energy harvesting. This allows us to obtain the updated state information for the next time step. (Including battery energy status and the status of each link channel).

[0118] In step 249 of some embodiments, after completing a virtual environment interaction, the current state information observed in that round is... The output of the current discrete action and the current continuous action and Current throughput reward feedback and the obtained update status information Packaged and integrated into a standard record information and record the information Continuous writing to a pre-configured experience replay pool with a capacity of C By storing empirical data in a database and using this mechanism of accumulating and randomly sampling empirical data, it is possible to effectively avoid the oscillation interference caused by continuous samples with high temporal correlation to the network gradient update.

[0119] In this application, the HPPO algorithm is described as follows.

[0120] The HPPO algorithm's policy update is based on the policy gradient theorem, using an advantage function to measure the merits of actions and provide direction for the gradient ascent of the actor network. Simultaneously, a pruning-based alternative objective function is used to constrain the policy update step size, ensuring training stability. This section first calculates the advantage function using Generalized Advantage Estimation (GAE), then derives the policy update objectives for discrete and continuous actor networks respectively, and finally presents the update criteria for the critic network.

[0121] Calculation of the dominance function based on GAE Advantage function Indicates the state Next action The difference between the return and the average return of following the current strategy in this state is calculated as follows: ; in, It is an action-value function, representing the state. Execute action The expected return is calculated directly. However, directly calculating the advantage function suffers from excessive variance, which can easily lead to oscillations in policy updates. Therefore, GAE is used to smooth the advantage function by introducing a discount factor. The core formula for balancing the bias and variance of the advantage function estimation is as follows: ; HPPO's discrete and continuous actor networks are updated independently, both employing pruning to replace the objective function, constraining the probability ratio of the new policy to the old policy within the range of [1-ε, 1+ε]. (This is the pruning factor) to avoid training failure due to excessively large policy update magnitudes. During training, set... and The parameters of the actor network are discrete and continuous, respectively. The old strategy of the wheel is (Discrete) and (Continuous), the first The new strategy for the wheel is (Discrete) and (Continuous), then the policy update objective of the discrete actor network is as follows: ; Among them, the new strategy Compared to the old strategy The probability ratio is used to measure the difference in action choices between the old and new strategies, as shown in the following formula: ; This represents the calculation of the mean of a finite number of samples. (Clipping factor) By using the pruning probability ratio, avoid The value should not be too large or too small to prevent drastic fluctuations in policy gradient updates and reduce training oscillations. This indicates that the value of the current action is higher than the average, and maximizing this formula will increase the value of the action. But it will not be allowed to exceed Conversely, if Maximizing this expression will reduce But it will not be smaller than . The coefficient is The policy entropy ensures the full exploration of the HPPO algorithm. Pruning function. The definition is shown in the following formula: ; Similarly, the policy update objective for the continuous actor network is shown in the following formula: ; The probabilities of the old and new strategies for consecutive actions are shown in the following formula: ; Two actor networks maximize their respective cropping / substitution objective functions using the gradient ascent algorithm, achieving iterative optimization of the strategy. In the HPPO algorithm, both the actor and critic networks employ mini-batch stochastic gradient descent for parameter updates. Specifically, during each update, C tuples are randomly selected from the experience replay pool. And update the parameters of the Actor network and Critic network. Therefore, the parameters of the discrete and continuous actor networks are... and The update can be expressed as the following formula: (k) (l) in, and represents the learning rate for discrete and continuous actor networks, respectively.

[0122] Value function update of the critic network During the training process, set These are the parameters for the critic network. Because in practical implementations, the advantage function... State value function in Since it cannot be calculated directly, we fit the actual cumulative discount reward, i.e., the target state value function. To accurately estimate the state value function The formula for calculating the target state value function is as follows: ; The update objective of the critic network is to minimize the temporal difference error, and the loss function is usually minimized using the mean square error function as shown in the following formula: ; For the commentator network, the parameters are also updated using mini-batch stochastic gradient descent. Therefore, the parameters... The update formula can be expressed as follows: (m) In step 2410 of some embodiments, based on the HPPO algorithm described above, when the cumulative time step reaches the update cycle, the system shuffles the experience replay pool and extracts a small batch of samples. In the specific network update calculation, the advantage function is first obtained by generalized advantage estimation to provide the direction of gradient ascent for the actor network. Then, the magnitude of a single update is limited by pruning to replace the objective function, and policy entropy is incorporated to ensure the sufficiency of environment exploration. On this basis, the parameters of the two actor networks are independently corrected. At the same time, the parameters of the initial critic network are updated by using temporal difference error. After multiple rounds of such gradient update cycles, the initial parameters will gradually converge to the optimal range and finally solidify into a transfer parameter optimization model that can be directly called.

[0123] The following is a schematic algorithm illustrating the training process of the initial transmission parameter optimization model in this application.

[0124] 1: Initialization: Discrete and Continuous Actor Network Parameters , , Commentator network parameters Experience replay pool The capacity C.

[0125] 2: For each round, e = 1, 2, ..., E do 3: Initialize the environment and obtain the initial state. ; 4: For each time step t = 1, 2, ..., T, do 5:

[0126] 5: Based on the status Actor online output strategy and Actions are obtained by sampling according to the strategy. = ; 6: The intelligent agent will take action The signal is transmitted to the RIS controller to adjust the switching closure, amplitude coefficient, and phase shift of each subarray of the RIS. 7: Action-based Solve problem (P1.2) to calculate the optimal receive beamforming vector at the base station; 8: The user sends a signal to the base station with the assistance of RIS and calculates the current reward. ; 9: Observe the status of the next time slot. ,get And save it to the experience replay pool middle; 10: if cnt%C==0 then 11: Use the experience replay pool Calculating the advantage function based on empirical information and target state value function ; 12: For epoch = 1,2,..., do 13: Disrupt the experience replay pool Experience information; 14: For f = 1,2,..., do 15: From Medium-sized small-batch experience information ; 16: Update network parameters according to formulas (k)(l)(m) , and ; 17: End For; 18: End For; 19: Clear experience data 20: End if; 20: End For; 21: End For; As shown in the algorithm above, during the initialization phase, the system first sets the initial parameters of the discrete actor network, continuous actor network, and critic network, and establishes an experience replay pool with a preset capacity. At each time step of each training round, the agent acquires the current environmental state and inputs it into two parallel actor networks. Through policy distribution sampling, a joint action is generated, including variables such as switch closure state, amplitude coefficient, and phase shift. Subsequently, the system issues this action to adjust the hardware state of the corresponding subarray and solves an optimization subproblem based on this action to obtain the optimal receive beamforming vector at the base station. After the communication node completes signal transmission and reception, the system calculates the immediate reward for the current time frame, observes the new state at the next time step, and continuously stores the transition tuple consisting of the state, action, reward, and new state into the experience replay pool.

[0127] When the time step count reaches the set capacity period of the experience replay pool, the algorithm triggers the gradient update mechanism for the neural network parameters. At this time, the system uses the historical experience information accumulated in the replay pool to uniformly calculate the advantage function and the target state value function. Then, within the set iteration period, the experience data is shuffled, and by extracting small batches of sample data, the weights of the discrete actor network, continuous actor network, and critic network are updated according to the corresponding update formulas. After completing this batch of parameter updates, the system clears the old data in the current experience replay pool so as to collect new policy interaction experience in subsequent time steps. Through continuous trial and error and iteration, the model converges and outputs the optimal transmission parameter optimization strategy.

[0128] Reference Figure 5 This is a schematic diagram of the model framework of the Hybrid Proximal Policy Optimization (HPPO) algorithm based on action combination provided in this application embodiment. In this framework, the system obtains the state information of the current time step from the communication environment consisting of a base station, a simultaneous transmission and reflection subarray reflector, and multiple users through an agent, and synchronously inputs this state information into the discrete actor network, continuous actor network, and critic network included in the algorithm model. The discrete actor network outputs a policy based on the input state information, obtaining discrete actions including mode switching, reflection phase shift, and transmission phase shift; the continuous actor network outputs a policy based on the input state information, obtaining continuous actions including amplification factor, power allocation factor, and terminal transmission power. Subsequently, the system performs lower-level solving based on the generated discrete and continuous actions, calculates the optimized receive beamforming vector on the base station side, and applies the above combined actions and beamforming vector together to the communication environment to adjust the physical operating state of the corresponding communication hardware, thereby obtaining the instantaneous throughput reward and the updated state of the next time step from the feedback of the communication environment. After completing a single environmental interaction, the system stores the experience information, which consists of the current state, action, reward, and updated state, in the experience replay pool. This information is then used to extract historical data samples during the network training phase to perform gradient iterative updates of the model parameters for the critic network, discrete actor network, and continuous actor network, thereby continuously optimizing the joint control strategy of system resources.

[0129] Through steps 246 to 2410 above, by combining independent action sampling with analytical solution of lower-level beamforming in each time step, and storing the generated quadruples of data in the experience replay pool, the strong correlation of the system's temporal evolution data is effectively broken, significantly improving the stability of the neural network during learning. Furthermore, by adopting a hybrid proximal policy optimization mechanism (HPPO) that includes generalized advantage estimation, pruning substitution objective function, and policy entropy, parallel gradient updates are performed on the discrete actor network, continuous actor network, and critic network. This effectively addresses the challenges of high action space dimension and strong coupling of discrete and continuous variables in communication scenarios assisted by simultaneous transmission and reflection subarrays, effectively preventing policy collapse and performance oscillations during training, and ensuring that the algorithm model can robustly converge to a better control policy. Thus, the communication system deploying this model has the efficient scheduling capability to cope with complex time-varying environments.

[0130] Through steps 210 to 240 above, by sequentially constructing the signal-to-noise ratio function and the average throughput target, the complex physical communication process is transformed into a rigorous parameter optimization problem. Furthermore, by employing a hierarchical optimization approach, the high-dimensional non-convex problem deeply coupled with discrete and continuous variables is decoupled into two sub-problems: an upper-level problem and a lower-level problem, effectively reducing the computational bottleneck of the optimization solution. Further, by establishing an initial transmission parameter optimization model based on a Markov decision process and introducing reinforcement learning for offline multi-round training, the final transmission parameter optimization model possesses powerful environment generalization and adaptive inference capabilities. In actual system deployment and online operation, it eliminates the need for time-consuming iterative solutions based on traditional convex optimization methods for each channel or energy change, significantly improving the resource allocation efficiency and real-time decision-making for simultaneous transmission and reflection subarrays.

[0131] Step 300: Solve based on the optimization parameters, the first channel state information, the second channel state information, the third channel state information, and the fourth channel state information to obtain the optimized reflection reception beamforming and the optimized transmission reception beamforming of the base station.

[0132] Step 300 is described in detail below.

[0133] In step 300 of some embodiments, the system uses a linear minimum mean square error detector to analytically solve the lower-level beamforming optimization subproblem (P1.2) based on the output optimization action variables and the channel state information of each link collected. This allows the system to calculate the optimal linear beamforming vectors for receiving user signals in the reflected and transmitted regions at the base station, thereby obtaining a better signal-to-interference-plus-noise ratio at the base station.

[0134] Step 400: Based on the optimized mode switch sequence, control the closed state of the mode switching switch of the corresponding subarray. Based on the optimized reflection phase shift sequence, optimized transmission phase shift sequence, optimized power distribution coefficient vector sequence and optimized amplification coefficient sequence, control the corresponding shared phase shift circuit, shared power distribution circuit and shared amplification circuit to coordinately regulate the transmission and reflection of the reflective surface of the simultaneous transmission and reflection subarray.

[0135] Step 400 is described in detail below.

[0136] In step 400 of some embodiments, the base station sends the determined action parameters to the controller of the reflector, which then performs coordinated control of the hardware circuitry at the physical level. The controller first controls the closed position of the single-pole double-throw switches within each subarray according to the optimized mode switch sequence to physically isolate or connect the internal circuits, thereby determining whether each subarray is actually operating in idle, passive, or active mode. After establishing the operating mode and the corresponding communication link, the controller further adjusts the phase response of the shared phase-shifting circuit using optimized reflection and transmission phase-shift sequences, adjusts the shunting ratio of the shared power distribution circuit using optimized power distribution coefficient vector sequences, and adjusts the gain of the shared amplifier circuit using optimized amplification coefficient sequences. Through the combined application of these multi-dimensional hardware parameters, the reflector can finely change the internal circuit connection logic, achieving coordinated physical control of the transmission and reflection characteristics of electromagnetic wave signals.

[0137] Step 500: Based on optimized reflection transmit power control, the first user terminal transmits first data to the base station; based on optimized transmission transmit power control, the second user terminal transmits second data to the base station; based on optimized reflection receive beamforming control, the base station receives the first data; and based on optimized transmission receive beamforming control, the base station receives the second data.

[0138] Step 500 is described in detail below.

[0139] In step 500 of some embodiments, after configuring the various hardware features of the reflector, the first user terminal and the second user terminal transmit uplink radio frequency signals containing service data into space according to the allocated optimized reflection transmission power and optimized transmission transmission power, respectively. Simultaneously, the base station applies pre-calculated optimized reflection reception beamforming and optimized transmission reception beamforming to perform targeted equalization and decoding processing on the superimposed electromagnetic signals propagating through direct links and cascaded links of the reflector in space, thereby accurately extracting and receiving the first and second data transmitted by each terminal in a multi-user interference environment.

[0140] Through steps 100 to 500 above, by jointly acquiring the real-time environmental channel state and battery energy state, and using a pre-trained parameter optimization model for action reasoning, the communication system can quickly and accurately decouple and calculate high-dimensional control parameters involving discrete and continuous variables, under the premise of satisfying the causal relationship of device battery energy and user rate constraints. This enables the system to schedule the operating energy consumption of each subarray more efficiently and with finer granularity under the condition of limited self-powered energy, thereby reducing the power consumption of the system's basic static circuit while improving the system's full-space data transmission capability and long-term average throughput.

[0141] To further verify the reliability of the parameter optimization method for the communication system proposed in this application, simulation experiments were conducted in the embodiments of this application, as described below.

[0142] Reference Figure 6 This is a schematic diagram of a simulation structure of a communication system provided in an embodiment of this application. Figure 6 The diagram illustrates the distribution of the base station, STAR-RIS, and users in a three-dimensional Cartesian coordinate system, with the coordinate axes in meters (m). In this embodiment, the coordinates of the base station and the center of STAR-RIS are set to (5m, 0m, 5m) and (0m, 20m, 10m), respectively. Without loss of generality, it is assumed that there are two users in the reflection area and two users in the transmission area of ​​the system. , The two users in the reflection area are evenly distributed on a circle with a radius of 10m and a center at (20, 100, 0); the two users in the transmission area are evenly distributed on a circle with a radius of 10m and a center at (-20, 100, 0). The path loss indices from the user to the base station, from the user to the STAR-RIS in the hybrid subarray architecture, and from the STAR-RIS in the hybrid subarray architecture to the base station are set to 3.2, 2.2, and 2.2, respectively. The path loss at a reference distance of 1 meter is set to -30dB, -20dB, and -20dB, respectively. It is assumed that the length of a single time frame is 1 second.

[0143] The settings for other simulation parameters in the system are as follows: , , , , , , , , , , Energy harvesting area , The phase shift quantization bits B of the reflection and transmission unit are 2, as shown in the following formula: ; The neural network structure and hyperparameters of the HPPO algorithm in the simulation are shown in the table below: Linear(m, n) represents a fully connected neural network layer with input dimension m and output dimension n; states_dim represents the dimension of the input states; disc_action and con_action represent the dimensions of the output discrete and continuous actions, respectively; Tanh and ReLU are activation functions. Unless otherwise specified, the parameters in the simulation remain unchanged. The temporal network structure of the HPPO algorithm is shown in the table below: ; The hyperparameter settings for the HPPO algorithm are shown in the table below: ; To evaluate the effectiveness of the proposed solution, the performance of the proposed solution will be compared with other benchmark solutions under different simulation settings. The detailed definitions of each solution are as follows.

[0144] Hybrid-Fully Connected STAR-RIS: Each STAR-RIS element in this architecture can freely switch between passive, active, and idle modes via a switch control; each element is equipped with an independent amplifier, power divider, and reflection and transmission phase shift controller.

[0145] Active-subarray STAR-RIS: In this architecture, the active STAR-RIS is divided into several subarrays, each of which shares an amplifier, power divider, and reflection and transmission phase shift controller.

[0146] Based on passive / active fully connected STAR-RIS: In this architecture, STAR-RIS is composed of... Sub-surfaces of active units and having The sub-surface of each passive unit is equipped with an independent amplifier, power divider, and reflection and transmission phase shift controller.

[0147] Based on active-fully connected STAR-RIS: Each element of the active STAR-RIS in this architecture is equipped with an independent amplifier, power divider, and reflection and transmission phase shift controller.

[0148] Based on passive-fully connected STAR-RIS: In this architecture, each element of the passive STAR-RIS is equipped with an independent power divider, reflection and transmission phase shift controller.

[0149] Reference Figure 7 This is a performance simulation diagram of the first method for optimizing the parameters of a communication system provided in this application. For example... Figure 7 The diagram shows the energy harvesting range. Number of STAR-RIS components At that time, the proposed scheme had the following subarray numbers: The convergence of the curves over time can be observed. It can be seen that all curves gradually converge, among which... The highest cumulative return is achieved over time. Secondly, and Performance drops significantly at 64 subarrays. This indicates that a larger number of subarrays is not always better; there is an optimal value. This is because a smaller number of subarrays limits the optimization freedom of STAR-RIS's amplification factor and reflection (transmission) phase shift, while too many subarrays increase hardware cost and energy consumption. Given the limited available energy of STAR-RIS, a moderate number of subarrays achieves a better balance between optimization freedom and energy consumption, thus resulting in superior system performance.

[0150] Reference Figure 8 This is a second performance simulation diagram of the parameter optimization method for the communication system provided in this application embodiment. For example... Figure 8 The diagram illustrates the effect of different numbers of STAR-RIS reflective (transmittance) elements. At that time, in the energy harvesting zone The trends of system average throughput with the number of subarrays are shown for the following conditions: [0,50], [50,100], [100,150], and [150,200]. Overall, based on the four energy harvesting scenarios, the system average throughput shows a continuous decreasing trend or an initial increasing followed by a decreasing trend, indicating that a larger number of subarrays is not necessarily better. This is because when... At this point, the system's available energy is very limited, and the number of subarrays capable of operating in active mode is also small. As the number of subarrays increases, the number of phase shift controllers and amplifiers in the system also increases accordingly, leading to a further increase in circuit power consumption. Due to the overall insufficient available energy, the effective energy allocated to signal reflection and amplification is further compressed. Increasing the number of subarrays can improve mode switching flexibility and the optimization freedom of reflection (transmission) phase shift and amplification factor, thus better utilizing limited energy resources under self-powered conditions. However, too many subarrays will also weaken their reflection (transmission) and amplification capabilities, reducing system throughput. This indicates that the proposed scheme can achieve a better balance between control freedom and subarray gain capabilities. In practical applications, subarray design should be rationally configured in conjunction with the energy state.

[0151] Reference Figure 9 This is a third performance simulation diagram of the parameter optimization method for the communication system provided in this application embodiment. For example... Figure 9The figure shows the relationship between the system's average throughput and the increase in average energy under different schemes, where the number of STAR-RIS components is... The number of subarrays involved in the subarray scheme is set to The horizontal axis represents the average energy across the energy collection intervals [0, 50], [50, 100], [350-400], [650, 700], [950, 1000], and [1250, 1300]. It can be seen that as the average energy increases, the average throughput of each scheme gradually improves. This is because more available energy provides sufficient energy support for user transmission and STAR-RIS operation, thereby enhancing the effective link quality and improving the system's data transmission capability. When the average energy is in the range of 25-675 mJ, the proposed hybrid-subarray STAR-RIS achieves a higher average system throughput than other schemes. When the average energy is greater than 675 mJ, the average throughput of the scheme proposed in this application is close to that of the active-subarray STAR-RIS scheme. This is mainly because under high energy budget conditions, system performance is no longer primarily limited by energy supply, but is more constrained by factors such as the upper limit of channel gain, array size, and the degree of freedom in resource allocation. At this point, the active units in the active subarray scheme can obtain sufficient energy support, and their active enhancement capabilities can be fully utilized, thereby narrowing the performance gap with the proposed scheme. On the other hand, the advantage of the proposed scheme is mainly reflected in the efficient utilization of system energy under energy-constrained conditions; as the energy supply gradually becomes sufficient, the marginal contribution of this energy efficiency advantage to throughput improvement weakens. Therefore, in the high-energy region, the two subarray-based schemes ultimately exhibit relatively similar performance upper limits. When the average energy exceeds 1275mJ, the hybrid-fully connected STAR-RIS slightly outperforms the scheme proposed in this application. This is because, under sufficient energy conditions, the system performance bottleneck no longer mainly stems from energy constraints, but depends more on the signal modulation freedom of the STAR-RIS. Compared to the subarray architecture, the fully connected architecture has higher degrees of freedom for phase shift and amplification factor optimization, enabling more refined joint optimization of each reflection / transmission unit.

[0152] Furthermore, for hybrid-fully connected STAR-RIS and active-subarray STAR-RIS, when available energy is limited, such as an average energy range of 25-75 mJ, hybrid-fully connected STAR-RIS can maintain system performance by controlling the operating mode of the reflection (transmission) units, relying on only some components to operate. Active-subarray STAR-RIS, however, operates in a fixed active mode. When available energy is limited, the limited energy must first meet the needs of the amplifier and phase shift controller, leaving less energy for effective signal enhancement, resulting in slower early performance improvements. As available energy increases, active-subarray STAR-RIS achieves faster throughput growth due to the lower circuit power consumption of its subarray structure. However, as the average energy further increases, hybrid-fully connected STAR-RIS, with its higher degree of freedom in optimizing phase shift and amplification factors, can better improve the system's average throughput.

[0153] This application also provides an electronic device, including: At least one memory; At least one processor; At least one program; The program is stored in a memory, and the processor executes the at least one program to implement the transmission parameter optimization method and transmission parameter optimization method of the communication system described above in this application. The electronic device can be any smart terminal, including mobile phones, tablets, personal digital assistants (PDAs), in-vehicle computers, etc.

[0154] Please see Figure 10 , Figure 10 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes: The processor 1001 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application. The memory 1002 can be implemented in the form of ROM (Read Only Memory), static storage device, dynamic storage device, or RAM (Random Access Memory). The memory 1002 can store the operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1002, and the processor 1001 calls and executes the transmission parameter optimization method and transmission parameter optimization method of the communication system of the embodiments of this application. Input / output interface 1003 is used to implement information input and output; The communication interface 1004 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 1005 transmits information between various components of the device (e.g., processor 1001, memory 1002, input / output interface 1003, and communication interface 1004); The processor 1001, memory 1002, input / output interface 1003 and communication interface 1004 are connected to each other within the device via bus 1005.

[0155] This application embodiment also provides a storage medium, which is a computer-readable storage medium, storing a computer program. When the computer program is executed by a processor, it implements the above-described method for optimizing transmission parameters of the communication system.

[0156] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0157] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0158] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0159] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0160] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0161] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. The coupling or direct coupling or communication connection between the shown or discussed units may be through some interfaces, or indirect coupling or communication connection between the apparatus or units, and may be electrical, mechanical, or other forms.

[0162] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0163] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0164] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0165] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A reflective surface for a simultaneous transmission and reflection sub-array, characterized in that, include: The system includes an energy harvesting circuit, a battery cell, and multiple subarrays. The energy harvesting circuit is connected to the battery cell and is used to harvest ambient energy and store it in the battery cell to power the multiple subarrays. Each of the subarrays includes a mode switching switch, a shared phase shifting circuit, a shared power distribution circuit, a shared amplification circuit, and multiple reflection and transmission units; In each of the subarrays, all the reflection and transmission units are connected to the shared phase-shifting circuit, and the mode switching switch is connected to the shared amplification circuit and the shared power distribution circuit respectively. The shared power distribution circuit and the shared phase-shifting circuit work together to achieve transmission and reflection modulation of the incident signal. When the simultaneous transmission and reflection subarray receives a mode control signal, the simultaneous transmission and reflection subarray is used to control the closing state of the corresponding mode switching switch based on the mode control signal, so as to control the corresponding reflection and transmission unit to switch between different working modes, and correspondingly change the communication link between the transmission modulation and the reflection modulation.

2. The simultaneous transmission and reflection sub-array reflector surface according to claim 1, characterized in that, The shared phase-shifting circuit includes a shared reflection phase-shifting circuit and a shared transmission phase-shifting circuit; in each sub-array, all the reflection and transmission units are connected to the shared power distribution circuit through the shared reflection phase-shifting circuit and the shared transmission phase-shifting circuit; the operating modes include idle mode, passive mode and active mode; In the passive mode, the mode switching switch directly connects the input incident signal to the shared power distribution circuit; In the active mode, the mode switching switch first amplifies the input incident signal through the shared amplifier circuit before inputting it to the shared power distribution circuit. The shared power distribution circuit is used to input the received incident signal to the shared reflection phase shift circuit and the shared transmission phase shift circuit respectively for phase modulation, so as to output the reflected signal and the transmitted signal simultaneously. In the idle mode, the mode switching switch is in the off state.

3. A parameter optimization method for a communication system, characterized in that, The communication system includes a base station, multiple first user terminals located in the reflection region, multiple second user terminals located in the transmission region, and a simultaneous transmission and reflection subarray reflector as described in claim 1. The method includes: The system acquires the first channel state information between each first user terminal and the base station, the second channel state information between each second user terminal and the base station, the third channel state information between the simultaneous transmission and reflection subarray reflector and the base station, and the fourth channel state information between each first user terminal and the second user terminal and the simultaneous transmission and reflection subarray reflector in the current time frame; and acquires the battery energy state information of the simultaneous transmission and reflection subarray reflector in the current time frame. The first channel state information, the second channel state information, the third channel state information, the fourth channel state information, and the battery energy state information are input into the transmission parameter optimization model for data processing to obtain optimization parameters. The optimization parameters include an optimization mode switching sequence, an optimization power allocation coefficient vector sequence, an optimization reflection phase shift sequence, an optimization transmission phase shift sequence, an optimization amplification coefficient sequence, an optimization reflection transmission power, and an optimization transmission transmission power. Based on the optimization parameters, the first channel state information, the second channel state information, the third channel state information, and the fourth channel state information, the optimized reflection reception beamforming and the optimized transmission reception beamforming of the base station are obtained. Based on the optimized mode switch sequence, the closed state of the mode switching switch of the corresponding subarray is controlled. Based on the optimized reflection phase shift sequence, the optimized transmission phase shift sequence, the optimized power distribution coefficient vector sequence, and the optimized amplification coefficient sequence, the corresponding shared phase shift circuit, the shared power distribution circuit, and the shared amplification circuit are controlled to coordinately regulate the transmission and reflection regulation of the reflective surface of the simultaneous transmission and reflection subarray. Based on the optimized reflection transmit power, the first user terminal is controlled to send first data to the base station; based on the optimized transmission transmit power, the second user terminal is controlled to send second data to the base station; based on the optimized reflection receive beamforming, the base station is controlled to receive the first data; and based on the optimized transmission receive beamforming, the base station is controlled to receive the second data.

4. The parameter optimization method for a communication system according to claim 3, characterized in that, The steps for constructing the transmission parameter optimization model include: Based on the reflection phase shift parameter sequence, transmission phase shift parameter sequence, amplification factor parameter sequence, power allocation factor parameter sequence, and mode switching parameter sequence of the reflective surface of the simultaneous transmission and reflection sub-array, as well as the transmission power parameters of each first user terminal and second user terminal and the beamforming parameters of the base station, a first decoding signal-to-noise ratio function for the reflection region and a second decoding signal-to-noise ratio function for the transmission region are constructed respectively. Based on the first decoding signal-to-noise ratio function and the second decoding signal-to-noise ratio function, the average throughput objective function is obtained, and a parameter optimization problem is constructed based on the average throughput objective function. Based on the beamforming parameters, the parameter optimization problem is divided into a lower-level beamforming optimization sub-problem and an upper-level resource joint optimization sub-problem. An initial transmission parameter optimization model is constructed based on the joint optimization sub-problem of the upper-layer resources, and the initial transmission parameter optimization model is trained in multiple rounds to obtain the transmission parameter optimization model.

5. The parameter optimization method for a communication system according to claim 4, characterized in that, The parameter optimization problem constructed based on the average throughput objective function includes: The total static circuit power consumption of the reflective surface of the simultaneous transmission and reflection subarray is obtained based on the static power consumption of the reflection phase shift control circuit, the static power consumption of the transmission phase shift control circuit, the static power consumption of the power divider, the static power consumption of the amplifier in the shared amplifier circuit, and the activation state of the operating mode of each subarray indicated by the mode switch parameter sequence. Based on the amplifier conversion efficiency, the fourth channel state information, the transmission power parameters, the power allocation coefficient parameter sequence, the amplification coefficient parameter sequence, and the input noise variance, the dynamic amplification power consumption of the simultaneous transmission and reflection subarray reflector is obtained. Based on the sum of the total static circuit power consumption and the dynamic amplification power consumption, the system energy consumption item for the current time frame is obtained; Based on the battery energy state information of the previous time frame, the environmental energy acquisition items of the current time frame, the system energy consumption items, and the maximum battery capacity, boundary constraints are calculated to obtain the battery energy update function; The parameter optimization problem is constructed based on the average throughput objective function and the battery energy update function.

6. The parameter optimization method for a communication system according to claim 5, characterized in that, The parameter optimization problem, constructed based on the average throughput objective function and the battery energy update function, includes: Obtain the current available battery energy, and based on the numerical relationship between the system energy consumption item and the current available battery energy, construct an energy causal constraint condition, wherein the energy consumed in the current time frame does not exceed the battery's stored energy at the beginning of the time frame. Based on the first decoding signal-to-noise ratio function and the second decoding signal-to-noise ratio function, the total amount of data uploaded by a single user in the reflection area and the transmission area are calculated respectively, and service quality constraints are constructed based on the numerical relationship between the total amount of data uploaded by a single user and the minimum service quality threshold. Based on the numerical relationship between the transmit power parameter and the maximum transmit power threshold, the numerical relationship between the amplification factor parameter sequence and the maximum amplification threshold, the numerical relationship between the power allocation factor parameter sequence and the power allocation factor range, and the matching relationship between the mode switch parameter sequence, the reflection phase shift parameter sequence, and the transmission phase shift parameter sequence and the discrete operating space, the physical constraints of the device are constructed. The parameter optimization problem is constructed based on the average throughput objective function, the battery energy update function, the energy causal constraint, the service quality constraint, and the device physical constraint.

7. The parameter optimization method for a communication system according to claim 4, characterized in that, The construction of the initial transmission parameter optimization model based on the joint optimization sub-problem of the upper-layer resources includes: Based on the first channel state information, the second channel state information, the third channel state information, the fourth channel state information, and the battery available energy state, a state space is obtained; A continuous action space is obtained based on the amplification factor parameter sequence, the power allocation factor parameter sequence, and the transmission power parameter; a discrete action space is obtained based on the mode switching parameter sequence, the reflection phase shift parameter sequence, and the transmission phase shift parameter sequence. Based on the average throughput objective function in the joint optimization subproblem of the upper-level resources, a throughput reward function is constructed. Obtain the initial parameters of the critic network, the initial parameters of the discrete actor network, and the initial parameters of the continuous actor network that outputs the joint continuous action. Based on the state space, the continuous action space, the discrete action space, the throughput reward function, the initial critic network parameters, the initial discrete actor network parameters, and the initial continuous actor network parameters, the initial transmission parameter optimization model is obtained.

8. The parameter optimization method for a communication system according to claim 7, characterized in that, The process of training the initial transmission parameter optimization model through multiple rounds of reinforcement learning includes: In each training time step, the current state information is input into the discrete actor network and the continuous actor network respectively, and the current discrete action and the current continuous action are obtained by policy distribution sampling respectively. The lower-level beamforming optimization subproblem is solved based on the current discrete action and the current continuous action to obtain the current receiving beamforming vector; The current discrete action, the current continuous action, and the current received beamforming vector are applied to the system training environment to obtain the current throughput reward calculated based on the throughput reward function, and to obtain the updated state information for the next time step. Based on the current state information, the current discrete action, the current continuous action, the current throughput reward, and the updated state information, record information is obtained and stored in the experience replay pool; When the preset update cycle is met, a small batch of samples is extracted from the experience replay pool, and gradient updates are performed on the initial critic network parameters, the initial discrete actor network parameters, and the initial continuous actor network parameters based on the advantage function, the pruning and substitution objective function, and the policy entropy calculated by the generalized advantage estimation.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the method for optimizing transmission parameters of the communication system according to any one of claims 3 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the method for optimizing transmission parameters of the communication system according to any one of claims 3 to 8.

Images