Method for parameter optimization of hybrid self-powered reflector communication system based on mobile user location information and related equipment

Abstract

This application proposes a parameter optimization method and related equipment for a hybrid self-powered reflector communication system based on mobile user location information. The communication system includes a transmitter, a hybrid intelligent reflector equipped with an energy harvesting circuit, and a mobile user. The method includes: acquiring the current location parameters of the mobile user at the current time step, and acquiring the current available energy of the hybrid intelligent reflector at the current time step; inputting the current location parameters and current available energy into a transmission parameter optimization model for data processing to obtain a target operating mode, a target phase shift, and a target amplification factor; in the current time step, adjusting the operating mode of each reflector unit in the hybrid intelligent reflector according to the target operating mode, adjusting the phase shift of each reflector unit according to the target phase shift, and adjusting the amplification factor of the reflector unit operating in active mode according to the target amplification factor, thereby improving the energy utilization efficiency and overall communication throughput performance of the communication system.

Description

Technical Field

[0001] This application relates to the field of wireless communication technology, and in particular to a parameter optimization method and related equipment for a hybrid self-powered reflector communication system based on mobile user location information. Background Technology

[0002] In recent years, Reconfigurable Intelligent Surface (RIS) technology, which reconstructs the electromagnetic wave propagation environment through numerous tunable components, has been regarded as a key enabling technology for effectively improving the spectral and energy efficiency of wireless communication systems. Traditional passive RIS suffers from severe multiplicative fading, which greatly limits communication performance; while deploying purely active RIS can amplify signals, it consumes additional energy and introduces amplified noise. To balance energy consumption and performance, the industry has proposed a hybrid RIS architecture that can flexibly operate in idle, passive, and active modes. In addition, considering that RIS often faces power bottlenecks in practical applications (such as the exterior walls of high-rise buildings) due to the inability to connect to the power grid and the high cost of battery replacement, configuring energy harvesting circuits for RIS to collect environmental energy (such as solar energy, radio frequency signals, etc.) to create self-powered communication systems has gradually become an important research direction for solving its practical deployment challenges.

[0003] In related technologies, parameter decisions for communication systems incorporating smart reflectors typically rely heavily on the acquisition of perfect instantaneous Channel State Information (CSI). However, due to the large number of reflector elements in smart reflectors, accurate channel estimation requires a massive amount of pilot symbols and communication resources, resulting in extremely high system overhead. Furthermore, estimation errors are easily generated in dynamic environments with mobile users, leading to a lack of practical deployability of related optimization schemes. Consequently, it is impossible to dynamically coordinate the switching of various reflector elements between idle, passive, and active modes, resulting in low energy efficiency and overall low communication throughput performance in existing communication systems with smart reflectors. Summary of the Invention

[0004] This application provides a parameter optimization method and related equipment for a hybrid self-powered reflective surface communication system based on mobile user location information, which can improve the energy utilization efficiency and overall communication throughput performance of the communication system carrying the intelligent reflective surface.

[0005] To achieve the above objectives, a first aspect of this application proposes a parameter optimization method for a hybrid self-powered reflector communication system based on mobile user location information. The communication system includes a transmitter, a hybrid smart reflector equipped with an energy harvesting circuit, and a mobile user. Each reflector unit of the hybrid smart reflector can switch between idle mode, passive mode, and active mode. The method includes: Obtain the current location parameters of the mobile user at the current time step, and obtain the current available energy of the hybrid smart reflector at the current time step; The current position parameters and the current available energy input transmission parameters are optimized by data processing to obtain the target operating mode, target phase shift and target amplification factor; In the current time step, the operating mode of each reflection unit in the hybrid intelligent reflective surface is adjusted according to the target operating mode, the phase shift of each reflection unit is adjusted according to the target phase shift, and the amplification factor of the reflection unit operating in the active mode is adjusted according to the target amplification factor.

[0006] To achieve the above objectives, a second aspect of this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the parameter optimization method for a hybrid self-powered reflective surface communication system based on mobile user location information as described in the first aspect.

[0007] To achieve the above objectives, a third aspect of this application provides a storage medium, which is a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the parameter optimization method for the hybrid self-powered reflector communication system based on mobile user location information described in the first aspect.

[0008] This application proposes a parameter optimization method and related equipment for a hybrid self-powered reflector communication system based on mobile user location information. The communication system includes a transmitter, a hybrid intelligent reflector equipped with an energy harvesting circuit, and a mobile user. Each reflector unit of the hybrid intelligent reflector can switch between idle mode, passive mode, and active mode. The method includes: first, obtaining the current location parameters of the mobile user at the current time step, and obtaining the current available energy of the hybrid intelligent reflector at the current time step; then, inputting the current location parameters and the current available energy into a transmission parameter optimization model for data processing to obtain a target operating mode, a target phase shift, and a target amplification factor; finally, in the current time step, adjusting the operating mode of each reflector unit in the hybrid intelligent reflector according to the target operating mode, adjusting the phase shift of each reflector unit according to the target phase shift, and adjusting the amplification factor of the reflector unit operating in active mode according to the target amplification factor. This application's embodiments replace traditional instantaneous channel state information (CSI) estimation by acquiring only the location parameters of mobile users, eliminating the massive pilot overhead and communication resource consumption caused by large reflector units, effectively avoiding channel estimation errors in dynamic environments, and greatly improving the practical deployability of the system optimization scheme. Simultaneously, this application's scheme combines the current location parameters of mobile users with the system's currently available energy and inputs this data into the optimization model for joint data processing and decision-making, overcoming the randomness of environmental energy harvesting and realizing flexible energy scheduling based on location awareness. Based on this decision result, the system can precisely control the dynamic switching of each reflector unit of the hybrid intelligent reflector between idle, passive, and active modes, and collaboratively optimize the amplification coefficients in phase shift and active modes. Thus, under the constraint of dynamically limited self-powered energy consumption, a balance between energy consumption and performance is achieved, improving the energy utilization efficiency of the communication system and effectively enhancing the overall communication throughput performance of the entire wireless communication system.

[0009] Other features and advantages of this application will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the application. The objectives and other advantages of this application may be realized and obtained by means of the structures particularly pointed out in the description, claims and drawings. Attached Figure Description

[0010] Figure 1 This is a schematic diagram of the structure of a hybrid self-powered reflective surface communication system provided in an embodiment of this application.

[0011] Figure 2 This is a flowchart of a parameter optimization method for a hybrid self-powered reflective surface communication system based on mobile user location information, provided in another embodiment of this application.

[0012] Figure 3This is another embodiment of the present application, which provides a training architecture for a deep reinforcement learning model based on the proximal policy optimization (PPO) algorithm and a schematic diagram of the interaction process between the agent and the environment.

[0013] Figure 4 This is a schematic diagram of the complete execution flow and interaction mechanism of a proximal policy optimization (PPO) algorithm provided in another embodiment of this application.

[0014] Figure 5 This is a schematic diagram of the hardware structure of an electronic device provided in another embodiment of this application. Detailed Implementation

[0015] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0016] It should be noted that although functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart.

[0017] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0018] In recent years, Reconfigurable Intelligent Surface (RIS) technology, which reconstructs the electromagnetic wave propagation environment through numerous tunable components, has been regarded as a key enabling technology for effectively improving the spectral and energy efficiency of wireless communication systems. Traditional passive RIS suffers from severe multiplicative fading, which greatly limits communication performance; while deploying purely active RIS can amplify signals, it consumes additional energy and introduces amplified noise. To balance energy consumption and performance, the industry has proposed a hybrid RIS architecture that can flexibly operate in idle, passive, and active modes. In addition, considering that RIS often faces power bottlenecks in practical applications (such as the exterior walls of high-rise buildings) due to the inability to connect to the power grid and the high cost of battery replacement, configuring energy harvesting circuits for RIS to collect environmental energy (such as solar energy, radio frequency signals, etc.) to create self-powered communication systems has gradually become an important research direction for solving its practical deployment challenges.

[0019] In related technologies, parameter decisions for communication systems incorporating smart reflectors typically rely heavily on the acquisition of perfect instantaneous Channel State Information (CSI). However, due to the large number of reflector elements in smart reflectors, accurate channel estimation requires a massive amount of pilot symbols and communication resources, resulting in extremely high system overhead. Furthermore, estimation errors are easily generated in dynamic environments with mobile users, leading to a lack of practical deployability of related optimization schemes. Consequently, it is impossible to dynamically coordinate the switching of various reflector elements between idle, passive, and active modes, resulting in low energy efficiency and overall low communication throughput performance in existing communication systems with smart reflectors.

[0020] To improve the energy efficiency and overall communication throughput performance of communication systems equipped with intelligent reflectors, this application replaces traditional instantaneous channel state information (CSI) estimation by acquiring only the location parameters of mobile users. This eliminates the massive pilot overhead and communication resource consumption caused by large reflector units, effectively avoiding channel estimation errors in dynamic environments and greatly improving the practical deployability of the system optimization scheme. Simultaneously, this application's scheme combines the current location parameters of mobile users with the system's current available energy and inputs this data into the optimization model for joint data processing and decision-making. This overcomes the randomness of environmental energy harvesting and achieves flexible energy scheduling based on location awareness. Based on this decision result, the system can precisely control the dynamic switching of each reflector unit of the hybrid intelligent reflector between idle, passive, and active modes, and collaboratively optimize the amplification coefficients in phase shift and active modes. Thus, under the constraint of dynamically limited self-powered energy consumption, a balance between energy consumption and performance is achieved, improving the energy efficiency of the communication system and effectively enhancing the overall communication throughput performance of the entire wireless communication system.

[0021] To better describe the parameter optimization method for the hybrid self-powered reflector communication system based on mobile user location information provided in this application, the hybrid self-powered reflector communication system applied to this parameter optimization method is first described below. (Refer to...) Figure 1 This is a schematic diagram of a hybrid self-powered reflective surface communication system provided in an embodiment of this application. Figure 1 As shown, this hybrid self-powered reflector communication system mainly consists of a transmitter, a self-powered intelligent reflector (RIS) equipped with energy harvesting circuitry, and a mobile user (MU) that moves randomly within a given area. Figure 1As shown, since the direct communication link is blocked by obstacles between the transmitter and the mobile user, a self-powered RIS (Radio Reflector) deployed in locations such as walls assists the transmitter in reflecting signals to the mobile user. Simultaneously, this RIS can harvest energy from the surrounding environment (such as solar energy) using its energy harvesting circuitry and use the harvested energy to power itself.

[0022] In addition, such as Figure 1 The diagram also details the hardware architecture design of the internal reflection unit of the hybrid RIS. Specifically, the RIS includes... Each reflective element integrates a phase-shifting circuit, a reflective amplifier, and two independent switches (a first switch and a second switch) for mode switching control. Furthermore, considering the practical scenario, it is assumed that the transmitter and RIS remain in fixed positions, while the MU can be positioned within a given area. Random variation within the region. The location is determined by the coverage area of ​​the RIS's reflected signal and the distribution of obstacles in the actual environment. The precise location information of the transmitter, RIS, and MU can be obtained using high-precision positioning technology.

[0023] In this application, it is assumed that and Let these represent the states of the two switches (i.e., the first switch and the second switch) inside the nth reflective element, where... The combination logic of these two switches allows for flexible switching of a single reflection unit between three operating modes: when the control switch state satisfies... When (i.e., the first switch is open), regardless of Why is this value determined when the nth reflective element operates in an extremely low-power idle mode? When the following conditions are met... and When the first switch is closed and the second switch is open, the reflective element n operates in passive mode, at which point only the phase-shifting circuit is activated to control the phase shift of the RIS; when the condition is met... and When both the first and second switches are closed, the reflective element operates in active mode. At this time, the phase-shifting circuit and the reflective amplifier are activated simultaneously, amplifying the incident signal while regulating the phase shift.

[0024] For the hybrid self-powered reflector communication system under consideration, the channel is set as a slow fading channel, and its coherence time is denoted as . For each time the channel remains constant, the correlation time... This is called a time slot.

[0025] When MU is in position When, record the time spent there as Without loss of generality, we assume that for all All have ,and For the mobile user's terminal MU located in the i-th time slot at position q, the channel coefficient vectors of the transmitter-to-RIS (TR) link, the RIS-to-MU (RU) link, and the transmitter-to-MU (TU) link are denoted as follows: , and .

[0026] In the hybrid self-powered reflector communication system under consideration, the transmitter sends information based on symbols. The duration of each baseband modulated symbol transmitted by the transmitter is denoted as . Because the system's channel is a slow fading channel, the channel remains unchanged during each baseband modulation symbol transmitted by the transmitter, thus having... In each time slot, the number of baseband modulation symbols transmitted by the transmitter can be expressed as shown in the following formula (1).

[0027]

[0028] When MU is in position Let S(i, j, q) be the j-th symbol transmitted by the transmitter in the i-th time slot, where Furthermore, assume that the system's CSI is unavailable, and only the MU's position information is obtainable. Therefore, the operating mode and phase shift control of each reflection unit in the RIS can only be based on the MU's position information. Let... Let be the phase shift matrix of RIS, where and These represent the states of the two switches in the nth reflective unit of MU at position q. This represents the amplification factor of the nth reflecting unit operating in active mode at position q. This represents the phase shift of the nth reflecting unit when the mobile user is at position q.

[0029] Based on the above description, when the MU is at position q, the transmitter sends the j-th symbol S(i, j, q) in the i-th time slot, and the symbol received by the MU can be represented as shown in the following formula (2).

[0030]

[0031] in, It is independent and identically distributed (iid) additive white Gaussian noise (AWGN).

[0032] In the hybrid self-functional reflector communication system of this application, the transmitter's transmit power is constant, denoted as . Because the time it takes for the transmitter to send one symbol is... Therefore, the energy required to send each symbol is Therefore, when MU is located at position q, its received signal-to-noise ratio (SNR) in the i-th time slot is as shown in the following formula (3).

[0033]

[0034] in It is the signal-to-noise ratio (SNR) of the transmitted signal. Here, B is the power spectral density of the AWGN, and B is the system bandwidth. When the transmitter's pulse shaping satisfies... When (for example, using a raised cosine pulse with a roll-off factor of 1), the received signal-to-noise ratio shown in formula (3) can be obtained as shown in the following formula.

[0035]

[0036] Therefore, the received bit signal-to-noise ratio of MU is , where M is the order used by the transmitter to modulate the transmitted information.

[0037] Based on such Figure 1 The hybrid self-powered reflector communication system shown below will be described in detail below, along with the parameter optimization method for the hybrid self-powered reflector communication system based on mobile user location information in the embodiments of this application. (Refer to...) Figure 2 This is an optional flowchart of a parameter optimization method for a hybrid self-powered reflector communication system based on mobile user location information provided in this application embodiment. Figure 2 The method may include, but is not limited to, steps 100 to 300. It is also understood that this embodiment... Figure 2 The order of steps 100 to 300 is not specifically limited; the order of steps can be adjusted or certain steps can be added or removed according to actual needs. This parameter optimization method for a hybrid self-powered reflector communication system based on mobile user location information can be applied to transmitters, processors, controllers, etc., connected to the hybrid self-powered reflector communication system.

[0038] Step 100: Obtain the current location parameters of the mobile user at the current time step, and obtain the current available energy of the hybrid smart reflector at the current time step.

[0039] Step 100 is described in detail below.

[0040] In step 100 of some embodiments, the current location parameters of the mobile user at the current time step are obtained, as well as the current available energy of the hybrid smart reflector at the current time step. In traditional smart reflector (RIS) assisted communication systems, the system typically needs to obtain accurate instantaneous channel state information (CSI) for parameter decision-making. However, due to the large number of reflector units, channel estimation consumes a massive amount of pilot symbols and communication resources, and is prone to estimation errors in mobile user mobility scenarios.

[0041] To address this technical challenge, the solution in this application employs a location-aware mechanism, directly acquiring the location parameters of the mobile user (MU) at the current time step (i.e., This implicitly characterizes large-scale fading and wireless channel quality status. Simultaneously, since the hybrid smart reflector in this application is equipped with an energy harvesting circuit for self-powering, the system must accurately grasp its energy status. The currently available energy (i.e., the energy obtained at this time) is... This does not refer only to the environmental energy collected at the current instant, but follows a cross-location energy scheduling protocol: the available energy at the current location equals the initial available energy at the previous location, minus the total energy consumed by the passive and active mode reflector units at the previous location, plus the environmental energy collected at the previous location. Its mathematical evolution expression is: This method of acquisition provides accurate physical boundary data for subsequent flexible allocation of global energy.

[0042] Step 200: Optimize the model by processing the current position parameters and the current available energy input transmission parameters to obtain the target operating mode, target phase shift, and target amplification factor.

[0043] Step 200 is described in detail below.

[0044] In step 200 of some embodiments, the current location parameters and the current available energy input transmission parameters are optimized using a data processing model to obtain the target operating mode, target phase shift, and target amplification factor. In this data processing and decision-making stage, the system uses the mobile user location parameters obtained in step 100... and currently available energy Combining and constructing into the system state space Subsequently, the state space data is fed into a pre-trained offline converged transfer parameter optimization model, which is essentially a deep reinforcement learning (DRL) model based on the proximal policy optimization (PPO) algorithm.

[0045] The actor network in the model evaluates and calculates the current input system state. Within a framework that strictly satisfies Quality of Service (QoS) and energy causality constraints, the long-term optimization objective is to maximize the system's average effective throughput. The output is an action space decision containing multi-dimensional action variables. Specific output technical parameters include: target operating mode parameters (corresponding to discrete switching variables in the mathematical model) used to indicate the switching of each reflection unit between idle, passive, or active modes. and The target phase shift (corresponding to discrete variable) used to reconstruct the electromagnetic wave reflection phase. ), and the target amplification factor specifically for the reflective units assigned to active modes (corresponding to continuous variables). ).

[0046] The transmission parameter optimization model is derived from a pre-constructed system optimization problem. The process of constructing the system optimization problem includes the following steps 110 to 140.

[0047] Step 110: The optimization objective is to maximize the average throughput of information received by mobile users within the mobile area.

[0048] Step 110 will be described in detail below.

[0049] In step 110 of some embodiments, the optimization objective is to maximize the average throughput of information received by the mobile user within the mobile area. In the performance evaluation of wireless communication systems, throughput is one of the most critical indicators for measuring the system's data transmission capability. Since the mobile user (MU) in this scheme is constantly moving within a given mobile area, its location parameters are constantly changing, which in turn causes dynamic fluctuations in the channel quality of the cascaded communication links from the base station to the reflector and from the reflector to the mobile user.

[0050] Therefore, simply optimizing parameters at a single instantaneous stationary position cannot truly reflect the overall communication performance of the system. To address this, this step establishes a macro-level optimization objective from a global perspective: maximizing the "average throughput" across the entire mobile area. In the specific mathematical model, this average throughput is expressed as the expected value of the average number of effective bits correctly decoded by the mobile user at each discrete location, which can be rigorously described as the objective function. ,in Let be the estimated bit error rate for the mobile user at location q, R be the system transmission rate, and T be the dwell time of the mobile user at each location. By establishing this global optimization objective, subsequent system parameter decisions can be guided to not only consider the real-time communication quality at the current location, but also to plan comprehensively from a macro perspective, thereby ensuring the overall communication transmission efficiency of the mobile user throughout the entire movement trajectory, as described below.

[0051] The optimization objective is to maximize the average throughput of information received by mobile users within the mobile area, including the following steps 111 to 115.

[0052] Step 111: Obtain the average bit error rate of information reception for mobile users at various locations within the mobile area.

[0053] Step 112: Subtract the average bit error rate of information reception from the first value to obtain the decoding accuracy.

[0054] Step 113: Based on the decoding accuracy, multiply by the transmission rate of the transmitter and the dwell time of the mobile user at each location to obtain the average number of bits that the mobile user can correctly decode at each location.

[0055] Step 114: Calculate the expected value of the average number of bits for all locations within the mobile area to obtain the expression for the average throughput.

[0056] Step 115: Use the expression for maximizing average throughput as the optimization objective.

[0057] Steps 111 to 115 are described in detail below.

[0058] In step 111 of some embodiments, the average bit error rate (BER) of information reception at various locations within the mobile area of ​​the mobile user is obtained. In actual mobile communication scenarios, the mobile user (MU) is in a state of constant motion, and the different locations it experiences within the mobile area (represented by the location parameter q) will cause varying degrees of large-scale fading and multipath effects in the wireless signal. In order to accurately evaluate the communication link quality at each physical location, the system needs to obtain the average bit error rate (BER) of the mobile user receiving transmitter signals at each location q, which is denoted as _____ in the mathematical model. The average bit error rate (BER) of received information is a key underlying communication performance metric. It directly reflects the probability of errors occurring in the data bits demodulated at the receiver, aided by the phase shifting and amplification strategies of the current hybrid intelligent reflector (RIS). By obtaining this BER parameter in a refined manner along the positional dimension, the system can provide the most fundamental data source for subsequently constructing a communication throughput model that closely reflects physical realities.

[0059] For a MU at position q, its bit error rate in the i-th time slot can be expressed as: For example, when using QPSK, the bit error rate of the MU in the i-th time slot is... ,in It is the complementary cumulative distribution function of the standard normal distribution. Furthermore, since the channels of the system vary randomly in different time slots, therefore... It is a random variable, which causes the bit error rate of the MU received information in each time slot. It also varies randomly. Therefore, this invention considers the average bit error rate of information reception at position q of the MU. Specifically, given the position q of the MU, the average bit error rate is defined as shown in the following formula (4).

[0060]

[0061] in This represents the mathematical expectation operation.

[0062] In step 112 of some embodiments, the decoding accuracy is obtained by subtracting the average bit error rate of information reception from the first value. In probability theory and communication principles, the state of data transmission can be binaryed as "error" and "correctness". The "first value" in this step mathematically represents the theoretical constant "1", that is, it represents an absolute probability of 100%. Specifically, the system subtracts the obtained average bit error rate of information reception from the constant 1. Its mathematical expression is embodied in The technical essence of this subtraction operation is to logically reverse the bit error rate, which was originally a "negative penalty indicator," into the decoding accuracy, which is a "positive benefit indicator." This decoding accuracy precisely characterizes the statistical probability that a mobile user, at a specific location q, can successfully recover and parse each received data bit without errors by the baseband chip, thus completing the mathematical and physical transformation from the underlying error probability to the effective information reception probability.

[0063] In step 113 of some embodiments, based on the decoding accuracy, the average number of bits that the mobile user can correctly decode at each location is obtained by multiplying the transmission rate of the transmitter and the dwell time of the mobile user at each location. In the physical layer transmission of a wireless communication system, the "transmission rate" (denoted as R) represents the number of data bits transmitted per unit time, while the "dwell time" (denoted as T) represents the time span or communication slot length of the mobile user at the current location q. Multiplying the transmission rate R by the dwell time T yields the total number of original data bits transmitted by the system at that location (i.e., RT). Subsequently, this total transmission amount is further multiplied by the obtained decoding accuracy, the physical meaning of which is to perform validity filtering on the total transmitted data, eliminating invalid data that has bit errors due to channel fading or noise interference. The final calculated product result This refers to the total amount of effective data that a mobile user can actually successfully receive and correctly decode at a specific location q, which is the effective throughput at that single location.

[0064] In step 114 of some embodiments, the mathematical expectation of the average number of bits corresponding to all locations within the mobile area is calculated to obtain an expression for the average throughput. Since the movement trajectory and location q of the mobile user throughout the mobile area have dynamic characteristics, focusing only on the effective number of bits at a single instant will not be sufficient to measure the long-term global performance of the communication system. Therefore, this step introduces a mathematical expectation operator from probabilistic statistics to perform a global statistical average of the correctly decoded average number of bits calculated for the mobile user at each discrete or continuous location. Through this expectation calculation operation, the system integrates and smooths the spatial communication performance of each local location from a macroscopic perspective, ultimately deriving a rigorous expression for the average throughput. This expression comprehensively considers the randomness of location changes, the physical rate of signal transmission, and the cumulative effect over time, representing a true mathematical mapping of the overall communication performance of the system.

[0065] In this application, the transmitter's information transmission rate is set to R bits / s. Since the dwell time of the MU at position q is T, the number of bits that the MU can correctly decode at position q is... The objective of this application is to maximize the MU's performance in a given region, given the Quality of Service (QoS) constraints at each location q. The average throughput during China Mobile's operation. The QoS constraint for the MU at each location q is defined as the average number of bits correctly decoded by the MU at each location q not being less than a given threshold. The Quality of Service (QoS) constraint is shown in the following formula (5).

[0066]

[0067] MU in the region The average throughput during medium-speed travel is defined as shown in the following formula (6).

[0068]

[0069] In step 115 of some embodiments, maximizing the average throughput expression is used as the optimization objective. After completing the transformation of the complex physical quantities of communication into rigorous mathematical expressions, this step establishes the final criterion for the parameter control of the entire system. The system places the derived expression within a maximization solution framework to construct a macroscopic optimization objective function. This optimization objective directly addresses the core business needs of mobile users—namely, acquiring as much effective, error-free data as possible throughout the entire mobile cycle. Once this objective is established, subsequent deep reinforcement learning models (such as the PPO algorithm) have a clear "reward / penalty evolution direction" when adjusting the working mode, phase shift, and amplification coefficient of the intelligent reflector, thereby driving the model to search for the optimal solution that enables the global data throughput to reach its theoretical peak from a massive number of parameter combinations.

[0070] Through steps 111 to 115 above, the underlying microscopic communication parameters (bit error rate, data rate, and time) are transformed into macroscopic system-level optimization objectives. This application's solution abandons the short-sighted limitations of traditional solutions that only optimize instantaneous signal-to-noise ratio or a single location, proposing a "global expected throughput" evaluation mechanism that integrates spatial movement trajectory and temporal cumulative effects. This modeling approach not only provides a robust and accurate mathematical analysis tool for handling highly random mobile channel environments but also ensures that subsequent transmission parameter optimization models always iterate along the correct direction of "maximizing the actual usable data volume for mobile users," which best meets the needs of practical engineering applications. Ultimately, this fundamentally guarantees the high efficiency and reliability of the self-powered hybrid intelligent reflector communication system.

[0071] Step 120: Use the operating mode parameters, phase shift parameters, and amplification coefficient parameters of each reflective unit of the hybrid intelligent reflective surface as optimization variables.

[0072] Step 130: Determine the quality of service constraints and energy causality constraints based on the operating mode parameters, phase shift parameters, and amplification factor parameters of each reflector.

[0073] Step 140: Based on the optimization objective, optimization variables, service quality constraints, and energy causality constraints, the system optimization problem is obtained.

[0074] Steps 120 to 140 are described in detail below.

[0075] In step 120 of some embodiments, the operating mode parameters, phase shift parameters, and amplification coefficient parameters of each reflector unit of the hybrid intelligent reflector are used as optimization variables. After clarifying the macroscopic optimization objective of maximizing average throughput, the system must determine the underlying physical control dimensions that can be actively adjusted to achieve this objective. Since this invention abandons the traditional single-characteristic reflector and adopts a novel three-mode hybrid intelligent reflector, its ability to reconstruct the radio electromagnetic propagation environment depends not only on the basic phase deflection but also on the flexible switching of the underlying hardware operating state and the power amplification of fading signals. Therefore, this step establishes the parameters of these three dimensions together as optimization variables: the first type is the "operating mode parameters" that control the on / off state of the hardware circuit, specifically corresponding to the discrete binary variables in the mathematical model that indicate the physical states of the first and second switches of the nth reflector unit. and The second category is the "phase shift parameter" that controls the phase of the reflected signal, specifically corresponding to discrete quantized phase shift variables. The third type is the "amplification factor parameter," which is specifically used to control the signal amplitude gain when the reflecting unit is in active mode, and specifically corresponds to a continuous variable. By jointly establishing these three types of parameters as optimization variables, the communication system is endowed with an extremely precise and flexible micro-physical resource scheduling space under limited energy.

[0076] In step 130 of some embodiments, service quality constraints and energy causality constraints are determined based on the operating mode parameters, phase shift parameters, and amplification coefficient parameters of each reflector. Any actual communication system engineering optimization cannot escape the limitations of physical boundary conditions, especially in self-powered scenarios where energy sources are limited. The values ​​of the multi-dimensional optimization variables set in step 120 must be strictly subject to the objective operating bottom line of the system. On the one hand, in order to ensure the most basic communication service experience for mobile users and prevent the communication link from being completely interrupted, the system establishes a "service quality (QoS) constraint" based on the optimization variables (as shown in formula (5) above). Mathematically, this constraint requires that when a mobile user receives a signal at each location, the average number of valid bits correctly decoded must not be less than a preset minimum service quality threshold. On the other hand, since the hybrid intelligent reflector is highly dependent on collecting environmental energy to maintain operation, the system further establishes an "energy causality constraint" (as shown in formulas (9) and (10) below). This constraint is the core physical law of self-powered scheduling. It strictly requires that the total energy consumed by the reflective units in the hybrid intelligent reflective surface, which are assigned to passive and active modes by variables, must never exceed the system's current available energy at its current location. Furthermore, the available energy when the system evolves to the next moving location must be strictly equal to the current available energy minus the aforementioned total energy consumption, plus the newly collected environmental energy at the current location. These two constraints together define a legal and safe physical boundary for the search of optimization variables.

[0077] As mentioned earlier, in Figure 1 In the hybrid self-powered reflector communication system shown, the RIS powers itself by harvesting energy from the external environment. Without loss of generality, it is assumed that the energy harvested by the RIS varies randomly. Since the RIS controls the operating mode and phase shift of its individual reflector units based on the position information of the MU, it is assumed that the energy harvested by the RIS when the MU is at its current position can only be used to move the MU to a subsequent position. When the RIS is at position q, the energy it has harvested is denoted as... .

[0078] for Figure 1The RIS-assisted communication system shown has two energy consumption components: one is the energy consumed by the reflection unit operating in passive mode, and the other is the energy consumed by the reflection unit operating in active mode. As mentioned earlier, the operating mode and phase shift control of each reflection unit of the RIS are based on the position information of the MU. Therefore, when the MU is at position q, the operating mode of each reflection unit of the RIS remains unchanged, and the energy consumed by the reflection unit operating in passive mode and active mode can be obtained as shown in the following formulas (7) and (8).

[0079]

[0080]

[0081] in, This represents the power consumption of the phase-shifting circuit in each reflecting unit. This indicates the power consumption of the reflective unit signal amplifier operating in active mode.

[0082] The energy consumed by the RIS must satisfy a causal constraint, meaning its energy consumption cannot exceed the energy it collects (i.e., energy causality constraint). Therefore, when the MU is at position q, the energy consumed by the RIS cannot exceed its available energy, i.e., the energy collected by the RIS when the MU was at the previous position. Furthermore, to enable the RIS to efficiently utilize the collected energy, the scheme in this application assumes that the energy consumed by the RIS at position q can be less than the energy it collects, and the unused remaining energy can be used when the MU moves to a subsequent position. In this way, when the MU and RIS are close, the channel quality between them is better, allowing the RIS to use more energy to assist information transmission and improve system throughput. Conversely, when the MU and RIS are far apart, the channel quality between them is poor, and the RIS can, under the condition of satisfying QoS constraints, reserve some available energy for the MU to use when it moves to a position closer to the RIS.

[0083] Based on the above RIS energy scheduling protocol, the energy consumed by RIS when MU is at position q should satisfy the following inequality (9).

[0084]

[0085] in Let be the initial battery energy of RIS when MU moves to position q. Also, based on the above analysis, we can obtain the initial battery energy (i.e., usable energy) of RIS when MU is at position q+1. It can be expressed as shown in the following formula (10).

[0086]

[0087] In step 140 of some embodiments, the system optimization problem is obtained based on the optimization objective, optimization variables, quality of service constraints, and energy causality constraints. This step is the final summary and logical closure of the underlying theoretical modeling of the system. By binding the mathematical objective function established in step 110, which aims to maximize the global average throughput, with the multidimensional optimization variables spanning the continuous and discrete domains determined in step 120, and by placing it rigorously within the boundaries of the quality of service threshold and the energy causality physical law established in step 130, a joint optimization problem representing the self-powered communication system is completely constructed at the mathematical level, as shown in the following problem (P1).

[0088]

[0089] Among them, constraint (1a) represents the QoS constraint, (1b) means that the energy consumed by RIS when MU is at position q cannot be greater than the available energy in its battery, (1c) is the battery energy evolution expression of RIS, (1d) represents the discrete value constraint of variables and, (1e) is the discrete value constraint of phase shift of each reflection unit of RIS, where L is the number of discrete phase shift values ​​of reflection units of RIS.

[0090] Because the solution space of this system's optimization problem not only mixes discrete switching and phase-shift variables with continuous amplification coefficient variables (exhibiting mixed-integer programming characteristics), but also intertwines a dynamic environment that constantly changes randomly with the location of mobile users and the state of energy harvesting (exhibiting stochastic programming characteristics), it is mathematically an extremely difficult mixed-integer nonlinear stochastic programming problem to solve directly using traditional convex optimization methods. The accurate formulation and construction of this system's optimization problem reveals the complex physical coupling relationships behind the system and provides an indispensable rigorous mathematical foundation and theoretical guidance for subsequently transforming the problem smoothly into a Markov Decision Process (MDP) and solving it using artificial intelligence algorithms for dimensionality reduction.

[0091] Through steps 110 to 140 above, a highly rigorous and physically realistic system optimization mathematical model was established. The proposed solution integrates the highly dynamic communication needs of mobile users with the highly stochastic environmental energy harvesting process within a unified and rigorous mathematical framework. By precisely defining three-dimensional optimization variables and dual hard constraints (QoS and energy causality), the complex physical coupling relationship between "target throughput improvement" and "limited available power scheduling" is thoroughly clarified. This not only fundamentally avoids the theoretical disconnect between traditional parameter optimization schemes and actual hardware energy consumption, but also paves the way for subsequent efficient model training using deep reinforcement learning algorithms and breaking through the bottleneck of stochastic programming solutions. This lays an extremely solid and reliable theoretical foundation for the final realization of low-power, high-reliability, and high-throughput operation of the hybrid intelligent reflector communication system.

[0092] Based on this, the process of constructing the transmission parameter optimization model includes the following steps 150 to 190.

[0093] Step 150: Obtain the state space based on the location parameters of the mobile user and the available energy parameters of the hybrid smart reflector.

[0094] Step 160: Based on the working mode parameters, phase shift parameters, and amplification coefficient parameters of each reflective unit in the hybrid intelligent reflective surface, obtain the action space.

[0095] Step 170: Based on the service quality constraints and energy causality constraints in the system optimization problem, construct a reward function model, which is used to maximize the effective throughput of the communication system.

[0096] Steps 150 to 170 are described in detail below.

[0097] In this application, to solve problem (P1) using the DRL method, the problem is first formulated as an MDP problem. Therefore, a quadruple needs to be defined. Here, the state space S represents the set of all possible states in the environment, the action space A represents the set of all possible actions the agent can take, the reward function r represents the immediate reward the agent receives after taking an action in a given state, and the discount factor reflects the degree of influence of the current state on future rewards, as described below.

[0098] In step 150 of some embodiments, the state space is obtained based on the location parameters of the mobile user and the available energy parameters of the hybrid intelligent reflector. When transforming a complex system optimization problem into a Markov Decision Process (MDP) for solution using deep reinforcement learning, it is first necessary to define the environmental state of the agent, i.e., the state space. In the hybrid self-powered reflector communication system of this application, there are two core physical environmental factors affecting system decision-making: the current spatial location of the mobile user (MU) and the current energy reserve of the system. Therefore, this step will determine the location parameters of the mobile user at time step t (…). The current available energy for hybrid intelligent reflectors (RIS) and hybrid intelligent reflectors (RIS) By combining these elements, the state-space expression under this MDP framework was precisely constructed. Among them, position parameters It implicitly characterizes the large-scale fading features of current wireless channels, thus perfectly replacing the transient channel state information (CSI) that is difficult to obtain in traditional schemes; while the available energy parameters This directly reflects the current energy boundary of the system, providing a rigorous physical premise for subsequent resource scheduling.

[0099] In the state space S of this application, the definition of a state should directly present key information about the current environment. For the communication system considered in this application, the agent makes decisions about the RIS operating mode, amplification factor, and phase shift by observing the position of the MU and the energy available at that position. Since large-scale fading of the wireless channel depends on the distance between the transmitter and receiver, while small-scale fading is considered unknown in this invention, the channel state information (CSI) of the link between the RIS and the MU can be indirectly reflected by the position of the MU. In other words, the time-varying position information of the MU directly characterizes the quality of the channel between the RIS and the MU and should be included in the state definition.

[0100] Furthermore, the energy collected by the RIS changes with the position of the MU. Since the RIS reflector consumes different amounts of energy when operating in different modes, the RIS will struggle to select the appropriate operating mode for each reflector if the available energy at the receiver's current position cannot be accurately determined. Therefore, at time step t, the system state is... Defined as the position of MU at this time step and its corresponding available energy, i.e.: .in, This indicates the position of MU at time step t. This represents the energy that MU can utilize at the corresponding location.

[0101] In step 160 of some embodiments, the action space is obtained based on the operating mode parameters, phase shift parameters, and amplification coefficient parameters of each reflective unit in the hybrid intelligent reflector. In the MDP model, the action space defines the set of all legal decision instructions that the agent can take after observing the current state. Corresponding to the optimization variables in the aforementioned system optimization problem, this step parameterizes the control instructions for the underlying hardware of the hybrid intelligent reflector into a specific action space. Specifically, the action space at time step t (denoted as...) It includes three dimensions of control variables: discrete switching parameters that control the switching of the working modes of each reflection unit (corresponding to the mathematical model in...). and Discrete phase shift parameters that control the reflected phase of electromagnetic waves (corresponding to) ), and the continuous amplification factor parameter for controlling the gain amplitude of the control signal in active mode (corresponding to By integrating these underlying hardware control signals, which contain both discrete and continuous variables, into a unified action space. The system provides a clear decision output dimension for deep reinforcement learning networks.

[0102] In the scheme of this application, action space A: an action is defined as a specific action or decision that an agent can take when it is in a certain state.

[0103] The goal of problem (P1) is to obtain the optimal operating mode, discrete phase shift, and amplification factor of the reflective units operating in active mode for each reflective unit of RIS at any position q within region Q. Therefore, at time step t, the action... Defined as .in, For the agent in state The decision made for the nth reflection unit of RIS at time step t.

[0104] In step 170 of some embodiments, a reward function model is constructed based on the service quality constraints and energy causality constraints in the system optimization problem. This reward function model is used to maximize the effective throughput of the communication system. In reinforcement learning algorithms, the reward function is the only feedback signal guiding the agent to learn the correct policy. To ensure that the training direction of the neural network remains absolutely consistent with the aforementioned rigorous system optimization problem, this step cleverly integrates the optimization objective and hard constraints into the design of the reward function.

[0105] The specific implementation logic is as follows: The system evaluates whether the agent's output action in a simulated environment satisfies both the "quality of service constraint" (i.e., the bit error rate or effective bit count meets the standard) and the "energy causality constraint" (i.e., the total energy consumption of the passive / active mode caused by the action does not exceed the currently available energy). If the action satisfies the above dual constraints, the simulated effective throughput generated by the action is given as a positive immediate reward value to encourage the agent to maximize throughput. Conversely, if the action causes any physical constraint to be broken (e.g., excessive power consumption or communication interruption), a very low penalty value is given or the reward value is directly set to zero. This constraint-penalized processing successfully transforms the communication optimization problem with strict physical constraints into an unconstrained reinforcement learning reward maximization problem, as described below.

[0106] Among them, a reward function model is constructed based on the service quality constraints and energy causality constraints in the system optimization problem, including the following steps 171 to 175.

[0107] Step 171: Based on the transmission symbols, cascaded channel model, and additive noise in the hybrid self-powered reflector communication system, obtain the analog received symbols corresponding to each reflector unit in each working mode.

[0108] Step 172: Based on the number of errors in the analog received symbols that were not correctly decoded and the total number of transmitted symbols, obtain the average symbol error rate, and based on the average symbol error rate divided by the logarithm of the modulation order, obtain the analog bit error rate.

[0109] Step 173: Subtract the simulated bit error rate from the first value, and then multiply it by the transmission rate and dwell time to obtain the simulated effective throughput model.

[0110] Step 174: When the simulated effective throughput model is greater than the preset threshold parameter, and the total energy consumed by the reflection unit in passive mode and active mode is not greater than the available energy parameter in the corresponding working mode, the reward function model is obtained based on the simulated effective throughput model.

[0111] Step 175: When the simulated effective throughput model is not greater than the preset threshold parameter, or the total energy consumed is greater than the available energy parameter in the corresponding working mode, obtain the reward function model based on zero.

[0112] Steps 171 to 175 are described in detail below.

[0113] In step 171 of some embodiments, based on the transmission symbols, cascaded channel model, and additive noise in the hybrid self-powered reflector communication system, the simulated received symbols corresponding to each reflector unit in each operating mode are obtained. During the offline training phase of the deep reinforcement learning model, the system needs to construct a simulated communication environment highly consistent with the real physical world within the computer. In this environment, the transmitter generates baseband transmission symbols for testing, which undergo complex channel fading as they propagate in space. Specifically, the signal passes through two spatial links: from the base station to the reflector, and from the reflector to the mobile user. The product of these two links constitutes the "cascaded channel model" of this step, i.e. Simultaneously, considering the actual receiver's internal thermal noise and environmental background interference, the system superimposes Gaussian white noise (i.e., additive noise) onto the signal. Combining the currently assigned idle, passive, or active operating modes of each reflector (and their corresponding phase shifts and amplification coefficients), the system performs mathematical convolution and superposition operations on the transmitted symbols, cascaded channel parameters, and noise, thereby reconstructing the analog received symbols that ultimately reach the mobile user after being controlled by the intelligent reflector array at the receiver. This step provides the most fundamental low-level simulation data for subsequent system performance evaluation.

[0114] In step 172 of some embodiments, the average symbol error rate (SER) is obtained based on the number of errors in the analog received symbols that were not correctly decoded and the total number of transmitted symbols. The analog bit error rate (BER) is then obtained by dividing the SER by the logarithm of the modulation order. After acquiring the analog received symbols, the system simulates the demodulation and decision-making process of the receiver, comparing the received symbols one by one with the original transmitted symbols sent by the transmitter. By counting the number of inconsistent symbols (i.e., the number of errors) and dividing it by the total number of transmitted symbols in that analog transmission, the system can calculate the SER. However, in modern digital communication, systems typically employ multi-level modulation techniques (such as M-QAM or M-PSK, where M is the modulation order), and a single symbol often carries multiple data bits. Therefore, to accurately measure the underlying information error probability, the system further divides the aforementioned SER by the base-2 logarithm of the modulation order (i.e., the base-2 logarithm of the modulation order). , representing the number of bits carried by each symbol, thus rigorously reducing the symbol-level error rate to the bit-level analog bit error rate (BER). This metric precisely quantifies the underlying data transmission reliability of the communication link under the current reflector parameter configuration.

[0115] In step 173 of some embodiments, the simulated bit error rate is subtracted from the first value, and then multiplied by the transmission rate and dwell time to obtain the simulated effective throughput model. This step aims to elevate the underlying bit error rate indicator to a macroscopic system communication performance indicator. Here, the "first value" represents the absolute probability constant "1" in the probabilistic mathematical model. The system uses the constant 1 to subtract the simulated bit error rate calculated in step 172 to obtain the probability that a single bit is correctly received (i.e., the decoding accuracy). Subsequently, the system multiplies this decoding accuracy by the physical transmission rate of the communication system (the number of bits sent per unit time) and the dwell time of the mobile user at the current location. The physical essence is that the transmission rate multiplied by the dwell time represents the original total amount of data sent by the system at this location; multiplying this by the decoding accuracy accurately eliminates invalid data damaged by fading and noise during transmission. The final calculated result is the total amount of correct information that the mobile user can actually obtain in this state, which is the simulated effective throughput model of this step.

[0116] In step 174 of some embodiments, when the simulated effective throughput model is greater than a preset threshold parameter, and the total energy consumed by the reflection units in passive and active modes is not greater than the available energy parameter in the corresponding working mode, a reward function model is obtained based on the simulated effective throughput model. In deep reinforcement learning algorithms, the reward function is the only feedback mechanism that guides the model to learn the optimal parameter configuration. Since the communication system of the present invention faces extremely strong physical boundary constraints, the system must convert these constraints into feedback signals that the model can understand. This step sets that if the reflection surface parameter configuration currently output by the model can make the calculated simulated effective throughput break through the preset service quality baseline (i.e., greater than the preset threshold parameter, satisfying the QoS constraint of uninterrupted communication), and at this time the total physical electrical energy consumed by all activated passive and active reflection units is strictly less than or equal to the current available energy parameter of the system (i.e., satisfying the energy causality constraint of self-powered energy), then this indicates that the action decision is legal and excellent. Under this compliant state, the system directly assigns the calculated simulated effective throughput value as a positive reward value to the model, thereby positively incentivizing the neural network to continuously optimize the network weight parameters in the direction of maximizing throughput while satisfying the constraints.

[0117] In step 175 of some embodiments, when the simulated effective throughput model is not greater than a preset threshold parameter, or the total energy consumed is greater than the available energy parameter in the corresponding working mode, a reward function model is obtained based on zero. Corresponding to the positive incentive in step 174, this step constructs an "absolute penalty mechanism" in the reinforcement learning environment. During the model's exploration of the parameter space, if the generated action instruction results in extremely poor communication quality (simulated effective throughput falls below or equals the preset threshold parameter), or if the action instruction overactivates the active amplifier, causing severe power overdraft (total energy consumed exceeds the current available energy parameter), it indicates that the action not only cannot guarantee communication, but is even an "illegal action" that cannot be executed in real physical hardware. At this time, the system will immediately impose a severe penalty, forcing its immediate reward value to zero. This hard penalty design of "zeroing out of bounds" can quickly reduce the proportion of such poor actions in the probability distribution of neural network decisions, forcing the actor network to quickly learn to avoid dangerous decisions that violate energy conservation and communication bottom lines.

[0118] In this application's scheme, regarding the reward model r: the reward is the core feedback indicator of the agent's interaction with the environment, used to reflect the quality of the agent's actions and guide it in learning the optimal policy. The objective function of problem (P1) is to maximize MU in the region. The average throughput during mobile operations. At time step t, the state is... At that time, MU was in position Therefore, the agent is in a state Next action The reward obtained is defined as shown in the following formula (11).

[0119]

[0120] in, When the agent fails to meet the minimum rate condition (1a) or energy consumption condition (1b) in the constraints of the optimization problem (P1) during training, the reward is defined as 0. This serves as a penalty for failing to meet the constraints of problem (P1), prompting the agent to learn actions that meet the constraints in order to obtain the maximum long-term reward, and ultimately learn the optimal policy.

[0121] In the process of training an agent, in order to obtain rewards , needs to be calculated .However, Since it lacks a closed-form expression, it cannot be computed analytically. To obtain... This application uses the Monte Carlo simulation method to... Make an estimate.

[0122] Therefore, a reliable estimate of the effective throughput of MU at the t-th time step can be obtained as shown in the following formula (12).

[0123]

[0124] Therefore, a reward can be obtained through (11). .

[0125] Through steps 171 to 175 above, a reward evaluation mechanism that seamlessly embeds complex physical constraints into the artificial intelligence training framework is constructed. This application's scheme, through rigorous communication link simulation (from symbol generation to throughput calculation), accurately reproduces the macroscopic benefits brought about by the hybrid intelligent reflector altering electromagnetic waves at the microscopic hardware level. More importantly, by introducing a dual threshold decision based on threshold and energy, and a "zero penalty for exceeding limits" design, the mathematically extremely difficult joint optimization problem with hard constraints is cleverly transformed into a standard, unconstrained reward maximization problem. This mechanism not only greatly accelerates the convergence speed of deep reinforcement learning models but also fundamentally ensures that the parameter strategy output by the model can be deployed absolutely safely and compliantly into real-world, constrained, self-powered communication hardware devices.

[0126] Step 180: Obtain the initial action parameters of the actor network and the initial comment parameters of the commenter network.

[0127] Step 190: Generate an initial transmission parameter optimization model based on the action space, state space, reward function model, initial action parameters, and initial comment parameters, and train the initial transmission parameter optimization model multiple times in a simulation environment to obtain the transmission parameter optimization model.

[0128] Steps 180 to 190 are described in detail below.

[0129] In step 180 of some embodiments, the initial action parameters of the actor network and the initial comment parameters of the commenter network are obtained. The deep reinforcement learning algorithm (such as the Proximal Policy Optimization algorithm, PPO) used in this invention is built on the mainstream "Actor-Critic" dual network architecture. The function of the "Actor Network" is to directly output the action policy based on the current input environmental state space (i.e., to determine how the reflex unit works); while the function of the "Critic Network" is to evaluate the quality of the current state or selected action of the actor network (i.e., to output a state value estimate). Before starting formal training, the weights and bias coefficients of these two deep neural networks must be initialized in computer memory. The "initial action parameters" obtained in this step are the random neural network weight parameters of the untrained actor network, and the "initial comment parameters" are the initial weight parameters of the untrained commenter network. These initial parameters constitute the starting point for the transfer parameter optimization model to perform self-learning and gradient descent iterations.

[0130] In step 190 of some embodiments, an initial transmission parameter optimization model is generated based on the action space, state space, reward function model, initial action parameters, and initial comment parameters. This initial transmission parameter optimization model is then trained multiple times in a simulated environment to obtain the optimized transmission parameter model. In this step, the state space, action space, and reward function constitute a complete reinforcement learning interaction logic. Combined with the initialized actor and commenter networks, they form an unoptimized initial transmission parameter optimization model. Subsequently, the system constructs a wireless communication "simulation environment" within the computer that matches the parameters of the real physical world. In this simulation environment, the initial model continuously observes the position and energy state of simulated mobile users, attempts to output various action combinations to control a virtual hybrid intelligent reflector, and continuously updates and adjusts the parameters of the actor and commenter networks using gradient descent mechanisms such as PPO based on the reward function values ​​fed back from the simulation environment. After thousands of episodes of closed-loop training involving "trial and error-feedback-update," the parameters of the neural network gradually converge to the optimal state. Finally, the maturely trained actor network is extracted as a "transmission parameter optimization model" that can be directly deployed and applied to actual communication base stations.

[0131] The PPO algorithm will be introduced in detail below.

[0132] Policy optimization (PPO) is an algorithm that directly optimizes the parameters of a policy to maximize cumulative reward. A policy is a rule or function that determines an agent's behavior in a specific state, defining... It represents the probability of an agent taking various possible actions under different states. In the PPO algorithm, there are two deep neural networks (DNNs): the Critic and the Actor. The Critic estimates the value function, guiding the Actor's policy update direction, while the Actor outputs the policy. By using the Critic and Actor in the PPO algorithm, the interaction between the agent and the environment (i.e., Figure 1 The hybrid self-powered reflector communication system shown can be based on Figure 3 The diagram in the image is used to describe this.

[0133] Reference Figure 3 This is a schematic diagram of a deep reinforcement learning model training architecture based on the proximal policy optimization (PPO) algorithm and the interaction process between the agent and the environment, provided in an embodiment of this application. Figure 3 As shown, the core of this training architecture consists of an agent comprising an Actor Network and a Critic Network, and a communication physical environment simulating real wireless propagation and energy constraints. During the closed-loop interaction of offline training, the simulation environment first inputs the system state space, representing the current mobile user location parameters and the available energy of the hybrid intelligent reflector, into the agent. The Actor Network performs forward computation based on this input state, outputting a combination of actions containing the discrete operating modes, discrete phase shifts, and continuous amplification coefficients of each reflector unit, and applies these actions to the simulation environment. After executing this set of action instructions, the simulation environment reconstructs the cascaded communication channel and calculates the corresponding effective system throughput based strictly on quality of service constraints and energy causality constraints, feeding it back to the agent as an immediate reward. Simultaneously, the environment itself outputs the updated state for the next time step over time. Subsequently, the system packages and stores these continuously generated "state-action-reward-update state" interaction sequences in an experience replay pool for caching. During the iterative update phase of the neural network parameters, the system randomly extracts batches of experience data from the experience replay pool, calculates the advantage function representing the merits of actions using the state value estimate output by the commentator network, and calculates the truncation importance weight by combining the importance sampling ratio of the old and new strategies and the preset truncation mechanism. Finally, the system calculates the gradient based on the constructed alternative objective function and backpropagates it to perform a smooth, safe, and constrained gradient update on the initial parameters of the actor network and the commentator network until the model parameters fully converge, thereby refining a transmission parameter optimization model that can be directly used for online control.

[0134] Based on such Figure 3 In the training architecture shown, at time step t, the agent first observes the position of MU and obtains the energy available at the current time step, thus obtaining the state. The Actor network is based on the state. Output the current policy The agent, based on the current policy Generate Actions And adjust the RIS phase shift via the RIS controller. Work mode and magnification factor This changes the wireless communication environment. When the RIS phase shifts... Work mode and magnification factor When adjusted, the transmitter sends L information bits to the MU via the RIS, and the MU estimates the effective throughput based on the received information bits and uses this to calculate the reward. After this, MU moves to the next position to generate the next state for the next time step. Ultimately, the agent can obtain the interaction data at the current time step. And store it in a buffer used to train the Critic and Actor.

[0135] To achieve optimal Actor and Critic through training, appropriate objective functions need to be defined for the two DNNs within the PPO framework. Therefore, the objective functions of PPO can be expressed as shown in equations (13) and (14) below.

[0136]

[0137] in, The parameters of the Actor network before each training round value and Let represent the probability ratio of the new strategy to the old strategy and the advantage function, respectively. Used to measure the difference in action choices between new and old strategies. The advantage function, representing the superiority of the current action compared to the average action, is estimated by the Critic. The goal of the Critic network is to accurately estimate the state-value function within the advantage function. The loss function is usually expressed as mean square error, as shown in formula (15).

[0138]

[0139] in, For Critic's network parameters, The target state value function is shown in the following formula (16).

[0140]

[0141] Among them, parameters This represents the Critic network parameters before each training round. .

[0142] Furthermore, to ensure that the policy performance is monotonically consistent, the proposed PPO method employs the PPO-Clip technique. Using the PPO-Clip technique, constraints are added to the objective function to ensure that the difference between the old and new policies does not become too large. The objective function of the PPO-Clip is defined as shown in Equation (17).

[0143]

[0144] in, Restricting x to [l, r], It is a hyperparameter that controls the amplitude limiting range. The coefficient is The strategy entropy can further improve the exploration capability of the PPO method. If This indicates that the value of the current action is higher than the average, which will increase... But it will not exceed Conversely, it will reduce But it will not be lower than .

[0145] Based on the above formula (17), the parameters are updated using the mini-batch stochastic gradient descent (SGD) method to maximize the objective function of PPO-clip. Specifically, F empirical data points are randomly drawn from the empirical replay pool, and then updated using the following formula. .

[0146]

[0147] in, This represents the learning rate of the Actor network. Furthermore, the mini-batch SGD method is used to update the parameters by minimizing the error term of the value function based on the above formula (15). It is estimated that the mean square error function is as shown in the following formula (19).

[0148]

[0149] in, This represents the learning rate of the Critic network.

[0150] Based on this, the training process of the initial transmission parameter optimization model in this application will be described in detail below, in conjunction with the above description.

[0151] The process of training the initial transmission parameter optimization model in a simulation environment for multiple rounds to obtain the transmission parameter optimization model includes the following steps 191 to 194.

[0152] Step 191: In the current training time step of each round of training, input the current training position parameters and the current available training energy into the actor network to obtain the training action distribution, and sample the current training action combination from the training action distribution.

[0153] Step 192: Execute the current training action combination in the simulation environment, obtain the current immediate reward based on the reward function model, and obtain the updated training state for the next training time step.

[0154] Step 193: Store the current training position parameters, current available training energy, current training action combination, current instant reward, and updated training status as training experience information in the experience replay pool.

[0155] Step 194: Iteratively update the initial action parameters and initial comment parameters using the batch training experience information sampled in the experience replay pool, and use the updated converged actor network as the transmission parameter optimization model.

[0156] Steps 191 to 194 are described in detail below.

[0157] In step 191 of some embodiments, at the current training time step of each training round, the current training position parameters and the current available training energy are input into the actor network to obtain the training action distribution, and the current training action combination is sampled from the training action distribution. In the actor-critic algorithm architecture of deep reinforcement learning, the actor network specifically undertakes the core function of "decision maker". At each time step of entering the simulation training, the system first feeds the combination of variables representing the current communication environment and physical boundary—that is, the current training position parameters of the mobile user and the current available training energy of the hybrid intelligent reflector—as input feature vectors to the input layer of the actor network. After nonlinear forward propagation calculation of the multi-layer neural network, the actor network does not directly output an absolutely certain and unique action instruction, but generates a probability distribution (i.e., training action distribution) covering all legal action spaces at the output layer. This probability distribution indicates the probability that the system will assign different working modes (idle, passive, active), phase shifts, and amplification coefficients to each reflector unit in the current "position-energy" state. Subsequently, the system performs random sampling based on the generated probability distribution to extract the current training action combination actually executed by the underlying hardware at the current time step. This probability distribution-based sampling mechanism is a key means of enabling artificial intelligence to "explore" actions in unknown communication environments. It ensures that the model can extensively try various possible combinations of three-mode hardware control parameters in the early stages of training, thereby effectively avoiding the training process from getting trapped in local optima too early.

[0158] In step 192 of some embodiments, the current training action combination is executed in the simulation environment, and the current immediate reward is obtained based on the reward function model, as well as the updated training state for the next training time step. Once the specific training action combination (i.e., including the precise switching mapping, phase modulation value, and signal amplification coefficient of each reflection unit) is determined through probabilistic sampling, the system sends these digital instructions to the underlying wireless communication simulation environment for physical and logical "execution". The simulation environment instantly reconstructs the virtual electromagnetic wave propagation channel based on these action parameters and rigorously calculates the actual physical impact of the action on the bit error rate of the cascaded communication link and the power consumption of the hardware battery. Subsequently, the system calls the aforementioned constructed reward function model, which integrates service quality constraints and energy causality constraints, to quantitatively evaluate the communication performance caused by the action: if the action is legal and compliant and effectively improves the throughput, a positive immediate reward is fed back to the model; if the action causes communication interruption or power depletion, a severe penalty of zero reward is imposed. Meanwhile, as the current time step naturally passes, the mobile user in the simulated environment will move along their trajectory to the next spatial coordinate, and the battery of the hybrid intelligent reflective surface will simultaneously calculate the energy consumption caused by this action and the newly collected environmental energy, thus feeding back to the system an updated training state for the next training time step, including the new location and new available energy. This step accurately and completely simulates the closed-loop feedback interaction process between intelligent decision-making and the constrained dynamic physical environment.

[0159] In step 193 of some embodiments, the current training position parameters, current available training energy, current training action combination, current immediate reward, and updated training state are stored as training experience information in the experience replay buffer. In continuous time-series decision-making tasks, the sequence data generated by the interaction between the agent and the environment often have strong temporal correlation. If the reinforcement learning model directly uses this highly correlated continuous data for real-time online parameter updates, it is very easy to cause drastic oscillations in the gradient update direction of the neural network, or even cause the model training to diverge and collapse. In order to completely solve this stability problem of algorithm convergence, this step introduces the core data management technology of "Experience Replay Buffer". Specifically, the system packages a complete interaction slice that occurred in the aforementioned steps, namely the initial state before the interaction (current training position parameters and available energy), the specific decision taken (current training action combination), the evaluation result obtained by the decision (current immediate reward), and the new state to which the environment transitions after the interaction (updated training state), into a standard data tuple, which is defined as an independent "training experience information". Subsequently, the system stores these continuously generated experience information entries one by one into an experience replay pool with pre-allocated storage space for silent caching. This experience replay pool acts as a vast historical memory bank during the agent's training process, specifically used to accumulate and aggregate massive amounts of environmental interaction samples, thus providing a rich data foundation for breaking data correlations and performing efficient and stable neural network backpropagation calculations.

[0160] In step 194 of some embodiments, the initial action parameters and initial comment parameters are iteratively updated using batch training experience information sampled from the experience replay pool, and the updated converged actor network is used as the transmission parameter optimization model. When the accumulated experience data in the experience replay pool reaches a preset scale, the system will formally start the self-evolution process of the neural network weights using a gradient descent optimization algorithm of deep learning (such as the core mechanism of the Proximal Policy Optimization (PPO) algorithm). The system no longer extracts data according to the chronological order of data generation, but instead randomly shuffles and extracts a small batch of training experience information from the experience replay pool. This random batch sampling operation completely eliminates the temporal correlation between training samples, ensuring the unbiasedness of gradient calculation and the smoothness of network parameter updates. In the specific iterative updates, the Critic Network uses this batch of experience information to calculate the prediction error of state value and updates its internal "initial Critic parameters" through backpropagation to continuously improve its accuracy in evaluating the quality of environmental states. Simultaneously, the Actor Network, guided by the advantage evaluation provided by the Critic Network, fine-tunes its "initial Action parameters" along the gradient ascent direction, increasing the probability of high-quality action combinations that can obtain high immediate rewards under constraints being selected in future decision probability distributions. After thousands of episodes of "sampling-evaluation-update" training, the parameters of both the Actor Network and the Critic Network eventually reach a stable convergence state without drastic fluctuations. Since the system only needs to quickly output action decisions based on the current state during the online deployment phase of a real physical communication base station without further complex value evaluation, after training, the Critic Network is separated, and only the updated and converged Actor Network is extracted, formally establishing it as a "transmission parameter optimization model" that can directly guide the operation of the hybrid intelligent reflector. The following will describe in more detail how the initial action parameters and initial Critic parameters are updated.

[0161] The process of iteratively updating the initial action parameters and initial comment parameters using batch training experience information sampled from the experience replay pool includes steps 1941 to 1946.

[0162] Step 1941: Use the commentator network to obtain the state value estimate corresponding to the training experience information.

[0163] Step 1942: Based on the current immediate reward and the state value estimate, calculate the advantage function model, which is used to characterize the degree to which the current training action combination is better than the average action.

[0164] Step 1943: Based on the probability ratio between the updated actor network policy and the original actor network policy, obtain the importance weight of the updated policy.

[0165] Step 1944: Perform numerical truncation on the importance weights of the update strategy based on the preset truncation range parameters to obtain the truncated importance weights.

[0166] Step 1945: Update the initial action parameters using gradients based on the product of the truncated importance weights and the advantage function model.

[0167] Step 1946: Update the initial comment parameters using gradients based on the mean square error between the target state value and the estimated state value.

[0168] Steps 1941 to 1946 are described in detail below.

[0169] In step 1941 of some embodiments, the state value estimate corresponding to the training experience information is obtained using the critic network. In the actor-critic architecture of deep reinforcement learning, the core technical function of the critic network is to evaluate the "goodness" or "potential" of the current system state. In this step, the system extracts training state space data containing the location of the mobile user and the available energy of the smart reflector from the experience replay pool and inputs it as a feature vector into the critic network. After forward propagation calculations within the network, the critic network outputs a scalar value, namely the state value estimate (usually mathematically denoted as...). This state value estimate represents the mathematical expectation of the long-term cumulative discounted reward that the system can expect to obtain in the current state if it continues to operate according to the current strategy into the future. This benchmark value provides an indispensable reference baseline for subsequent accurate measurement of the actual physical benefits of specific actions.

[0170] In step 1942 of some embodiments, an advantage function model is calculated based on the current immediate reward and the state value estimate. This advantage function model characterizes the degree to which the current training action combination is superior to the average action. Simply knowing how much immediate reward a particular action combination receives is insufficient; the system needs to determine whether the action is better or worse than the conventional policy. Therefore, this step introduces the advantage function (usually denoted as...). As shown in formula (14) above). In specific calculations, the system utilizes the current instantaneous reward fed back from the simulated environment ( ), plus the discount factor ( The estimated value of the next state after decay ( ), and then subtract the estimated value of the current state ( ), that is, mathematical expression That is, similar to the formula (14) above. The calculated advantage function model accurately quantifies the additional throughput gain brought by "the specific hardware control action taken now" compared to "the average expected action that the system usually takes in this state". If the advantage function is positive, it means that the combination of the currently assigned working mode, phase shift and amplification coefficient is extremely advantageous and its output probability should be increased in subsequent training; if it is negative, it means that the combination of actions is inferior to the average level and needs to be suppressed.

[0171] In step 1943 of some embodiments, the importance weight of the updated policy is obtained based on the probability ratio between the updated and unupdated actor network policies. To improve the efficiency of training data utilization, the Proximal Policy Optimization (PPO) algorithm of this application allows the use of historical interaction data generated by the "old policy" in the experience replay pool to update the current "new policy" (i.e., offline policy update). However, since the probability distribution of the same action generated by the old and new policies has changed, directly using the old data can lead to severe gradient bias. Therefore, this step introduces an importance sampling technique. The system calculates the training action combination at the current time step in the updated actor network (new policy). ) and the pre-updated actor network (old strategy) The probability of being selected under each policy is calculated by dividing the two probabilities. This probability ratio is the importance weight of the update policy. Mathematically, it rigorously compensates for the differences in data distribution under different policies, ensuring the unbiasedness of gradient estimation.

[0172] In step 1944 of some embodiments, the importance weights of the update strategy are numerically truncated based on a preset truncation range parameter to obtain truncated importance weights. Although the importance weights in step 1943 address the distribution bias problem, if the difference between the old and new strategies is too large, the weights can become extremely disparate, causing severe oscillations in the network during gradient updates, or even completely destroying the learned, excellent communication parameter strategies. To restrict policy updates to a safe "trust region," this step introduces a pruning mechanism. The system sets a small preset truncation range parameter (hyperparameter). (For example, usually taken as 0.2), the importance weights calculated in step 1943 are forced to be applied using the clipping function. Limited to the range to Within this range, as shown in formula (17) above. The truncation importance weights output after this hard numerical truncation process essentially set a strict physical boundary for subsequent network parameter iterations, ensuring that the system performance will only increase monotonically and steadily, and will not experience a catastrophic drop.

[0173] In step 1945 of some embodiments, the initial action parameters are updated by gradient based on the product of the truncated importance weights and the advantage function model. This step is the final execution stage for the actor network (i.e., the decision network that outputs specific reflector hardware instructions) to complete its self-evolution. The system constructs a trimmed alternative objective function (Surrogate Objective Function) in the PPO algorithm, which takes the minimum value between the product of the untruncated weights and the advantage function and the product of the truncated importance weights and the advantage function. Subsequently, the system solves the partial derivative of the objective function with respect to the initial action parameters and updates the weight matrix of the neural network using the stochastic gradient ascent algorithm, as shown in the above formula (18). Through this iterative update process, the actor network is precisely fine-tuned: it will be more inclined to output reflector unit working modes and phase shift parameter combinations with high positive advantage (i.e., able to significantly improve throughput under energy constraints), while the update magnitude of its strategy is tightly constrained within the safe physical boundary defined by the truncated range.

[0174] In step 1946 of some embodiments, the initial comment parameters are updated using gradients based on the mean squared error between the target state value and the state value estimate. As the actor network evolves, the commenter network responsible for evaluating value must also improve the accuracy of its predictions. The system first calculates the target state value, which is typically composed of the current real immediate reward plus the predicted value of the next state, representing a "label" that is closer to the feedback of the real environment. Subsequently, the system calculates the mean squared error (MSE) between the target state value and the "state value estimate" originally output by the commenter network in step 1941, and uses this as the loss function of the commenter network, as shown in formula (19) above. The system applies the stochastic gradient descent algorithm to this mean squared error loss function, backpropagates, and updates the initial comment parameters. This process enables the commenter network to predict the potential system throughput value of mobile users in various locations and energy states more and more accurately after multiple iterations, thereby providing the actor network with more accurate and reliable advantage evaluation guidance.

[0175] Through steps 1941 to 1946 above, a highly efficient and stable closed-loop PPO deep reinforcement learning algorithm is achieved. The scheme in this application cleverly combines importance sampling and trust region truncation techniques, not only completely solving the pain point of low data sample utilization in traditional reinforcement learning algorithms, but also fundamentally avoiding policy collapse caused by excessively large network weight update steps. This enables the hybrid intelligent reflector parameter optimization model, which contains a large number of discrete switching variables and continuous amplification coefficients, to converge smoothly and rapidly to the global suboptimal or optimal solution within extremely limited self-powered energy constraints, with a monotonically increasing trend. Ultimately, this ensures the ultimate reliability and robustness of artificial intelligence decision-making technology in complex practical communication engineering applications.

[0176] Reference Figure 4 This is a schematic diagram illustrating the complete execution flow and interaction mechanism of a proximal policy optimization (PPO) algorithm provided in an embodiment of this application. Figure 4 As shown in the algorithm flow, in the t-th time step of deep reinforcement learning training, the algorithm begins with the agent's observation of the environment. The agent observes the current position of the mobile user (MU) and the energy available to the hybrid intelligent reflector (RIS) in the t-th time step, thereby obtaining the current system state. Based on the obtained state The network of actors within the intelligent agent, through the current policy Perform random sampling to obtain specific action instructions. The system then sends the action command to the RIS controller of the underlying physical device. Based on the received action command... The RIS controller precisely adjusts the operating mode (idle, passive, or active), phase shift, and amplification factor in active mode of each reflector, thereby reconstructing and controlling the actual radio electromagnetic propagation environment.

[0177] In the wireless environment controlled and reconfigured by RIS, the system enters the environmental interaction and performance evaluation phase. First, the transmitter transmits L bits of information to the mobile user (MU). The mobile user, at the receiving side, uses the Monte Carlo method based on formula (12) to simulate and calculate the given effective throughput. Since the MU has already calculated the effective throughput, it can feed it back to the transmitter through the feedback link; subsequently, the transmitter will calculate the instantaneous reward corresponding to the current action based on formula (11), combined with service quality and energy constraints. Upon receiving the reward Subsequently, as time progresses, the mobile user physically moves to the next spatial location, and the system obtains the updated state for the next time step. .

[0178] Once the complete interaction of this time step is finished, the agent stores the sequence of transition data obtained from the interaction between the transmitter and the environment (i.e., data tuples containing state, action, reward, and next state) uniformly in the experience replay pool B. As the interactions of time steps are completed, the experience replay pool accumulates sufficient samples. The system calculates the advantage function and target state value function for each transition data in the experience replay pool B using formulas (14) and (16), respectively. Finally, the algorithm enters the network parameter update stage, where the system randomly samples a mini-batch of data from the experience replay pool B to train the Critic and Actor networks. It is particularly noteworthy that, as shown in steps 16 to 22 of Algorithm 1, each sample in the training is repeatedly used to train the Critic and Actor up to 1600 times. This design greatly improves the efficiency of sample utilization because, through repeated deep training on the same samples, the agent can learn the complex communication data features more fully and thoroughly, thereby significantly improving the overall training efficiency and throughput optimization performance with limited interaction data.

[0179] Through steps 191 to 194 above, a complete, closed-loop, and highly automated deep reinforcement learning offline training engine was constructed. The proposed solution fully combines the decision-making advantages of the actor-commentator dual-network architecture with the training stability of the experience replay mechanism. This allows the system to deeply learn and internalize the extremely complex multidimensional nonlinear coupling relationship between "mobile user movement trajectory - battery energy scheduling - physical three-mode mapping" through massive virtual "trial and error" and gradient evolution, without consuming real network baseband resources or affecting existing mobile user communication. This not only fundamentally overcomes the technical bottleneck that hybrid stochastic programming problems with stringent physical boundary constraints are difficult to solve effectively using traditional mathematical algorithms, but also the final refined actor network model possesses extremely low computational complexity and millisecond-level forward inference capability. This makes the highly challenging self-powered hybrid intelligent reflector parameter joint optimization scheme truly feasible for low-latency online deployment and commercial application in actual mobile communication networks.

[0180] Through steps 150 to 190 above, the solution of this application cleverly reduces the dimensionality of the "nonlinear, mixed integer, and highly stochastic" system optimization problem, which was originally proven to be extremely difficult to solve directly in mathematics, and transforms it into a standard Markov decision process. Relying on the Actor-Critic architecture and massive trial-and-error training in a simulation environment, the model internalizes the extremely complex physical coupling relationship of "location-energy-throughput". This offline training and online inference architecture design enables real-world communication systems to avoid the need for time-consuming traditional iterative optimization calculations when facing rapidly moving mobile users and rapidly changing self-powering states. Instead, by simply inputting the current parameters into the trained model, highly reliable hardware control commands can be directly output within milliseconds, thus completely breaking down the technical barriers between complex theoretical optimization and efficient deployment in practical engineering.

[0181] Step 300: In the current time step, adjust the working mode of each reflection unit in the hybrid intelligent reflective surface according to the target working mode, adjust the phase shift of each reflection unit according to the target phase shift, and adjust the amplification factor of the reflection unit in the active mode according to the target amplification factor.

[0182] Step 300 is described in detail below.

[0183] In step 300 of some embodiments, in the current time step, the operating mode of each reflective unit in the hybrid smart reflector is adjusted according to the target operating mode, the phase shift of each reflective unit is adjusted according to the target phase shift, and the amplification factor of the reflective unit operating in active mode is adjusted according to the target amplification factor. This step is the precise execution process of system decisions at the underlying physical hardware level. After receiving the decision command output in step 200, the controller of the hybrid smart reflector will change the wireless propagation environment of the system through microcircuit control.

[0184] The controller first precisely switches the corresponding reflective unit to idle mode (circuit disconnected to achieve zero power consumption), passive mode, or active mode based on the on / off status indication of the target operating mode. For reflective units activated in passive mode, the controller only supplies power to their phase shift circuit and strictly follows the target phase shift parameters. The phase deflection angle is adjusted to achieve basic spatial beamforming; for the reflector unit activated in active mode, the controller simultaneously supplies power to its phase shifting circuit and reflective amplifier, and under the same frequency synchronization of phase adjustment according to the target phase shift parameters, strictly according to the target amplification factor. The amplitude of the incident weak electromagnetic wave signal is amplified, thereby directly compensating for severe path multiplicative fading.

[0185] Throughout the entire physical adjustment and signal reflection process, the system strictly ensures the power consumption of the reflection unit in passive mode from the hardware level. Energy dissipation of the reflective unit in active mode The sum must never exceed the currently available energy obtained. That is, satisfying .

[0186] The process of adjusting the working mode of each reflective unit in the hybrid intelligent reflective surface according to the target working mode includes the following steps 310 to 320.

[0187] Step 310: When the target working mode is idle mode, control the first switch to turn off to shut down the corresponding reflection unit.

[0188] Step 320: When the target operating mode is passive mode, control the first switch to close and control the second switch to open to activate the phase shifting circuit.

[0189] Step 330: When the target operating mode is active mode, control both the first switch and the second switch to close simultaneously to activate the phase shift circuit and the reflective amplifier.

[0190] Steps 310 to 330 are described in detail below.

[0191] In step 310 of some embodiments, when the target operating mode is idle mode, the first switch is turned off to shut down the corresponding reflective unit. Specifically, each reflective unit of the hybrid smart reflective surface (RIS) is internally configured with a switching component for controlling the hardware connection state, including a first switch (i.e. The corresponding switch) and the second switch (i.e. (Indicates the corresponding switch). When the transmission parameter optimization model's decision for the nth reflection unit is to require it to enter idle mode, the system controller will directly output a control signal, causing the first switch of that reflection unit (i.e., satisfying the condition) to activate. The second switch is in a physically disconnected state. Under this switching logic, regardless of whether the second switch ( Regardless of whether the reflective unit is in a closed or open state, no operating power is fed into the internal circuitry, thus achieving complete shutdown of the unit. This hardware-level physical disconnection mechanism ensures that the reflective unit allocated to this mode has extremely low static power consumption (i.e., near-zero power consumption). This maximizes the conservation of valuable environmental energy harvesting capacity, strictly meeting energy causality constraints, even when the system's available energy is extremely limited or the reflective unit contributes very little to improving the quality of current mobile user communication links.

[0192] In step 320 of some embodiments, when the target operating mode is passive mode, controlling the first switch to close and controlling the second switch to open activates the phase-shifting circuit. Specifically, when the optimization model determines that the current reflector unit needs to use basic electromagnetic wave phase modulation to assist communication, it sets it to passive mode. At this time, the system controller issues a hardware control command to switch the first switch of the corresponding reflector unit to the closed state (i.e., satisfying...). At the same time, the second switch remains in the open state (i.e., satisfying the condition). Through this specific combination of switching states, the phase-shifting circuit inside the reflective unit is officially powered on and activated, while the power supply line to the reflective amplifier is blocked. The activated phase-shifting circuit is a typical low-power component, and its main technical function is to strictly adhere to the target phase-shift parameters determined in the aforementioned steps (i.e., This method performs a predetermined phase deflection or reconstruction on the electromagnetic wave signal incident on the surface of the reflective unit, thereby changing the spatial direction of the reflected beam to align with the mobile user and improve signal reception quality. This mode achieves significant spatial diversity gain and beamforming effects with relatively low base energy consumption (consuming only the maintenance power of the phase-shifting circuit), representing the core operating state where the system seeks a fundamental balance between strict energy consumption constraints and communication performance.

[0193] In step 330 of some embodiments, when the target operating mode is active mode, both the first and second switches are closed to simultaneously activate the phase-shifting circuit and the reflective amplifier. Specifically, in scenarios with extremely harsh communication environments, where mobile users are far from the reflector causing severe spatial path loss, or where the system currently has ample available energy, the optimization model will allocate some reflective units that play a key relay role to active mode. In this mode, the system controller will drive the first switch (i.e., satisfying...) ) and the second switch (i.e., satisfying Simultaneously, the circuit closes. This hardware linkage not only activates the aforementioned low-power phase-shifting circuit, but more importantly, it synchronously powers and activates the high-power reflective amplifier integrated within the reflective unit. Thus, when an electromagnetic wave strikes the reflective unit, it undergoes not only precise spatial phase adjustment via the phase-shifting circuit but also substantial physical power amplification via the reflective amplifier. The activated reflective amplifier will strictly adhere to the target amplification factor output by the optimized model (i.e.,...). This effectively amplifies the amplitude of the incident signal, thereby directly and powerfully compensating for the severe "multiplicative fading" experienced by the wireless signal in the two cascaded links from the base station to the reflector and from the reflector to the mobile user. Although enabling this active mode incurs a relatively high energy consumption burden (requiring simultaneous power supply to the phase-shifting circuit and the amplifier circuit), its effect on overcoming system transmission bottlenecks, improving the signal strength at the receiving end, and reducing the bit error rate is the most direct and significant.

[0194] Through steps 310 to 330 above, a precise, low-latency, and highly efficient "three-state" hardware switching logic is established. The solution in this application achieves smooth switching and physical closed-loop operation of the hybrid intelligent reflector between low-power sleep (idle mode), basic phase modulation (passive mode), and high-energy-consumption signal active amplification (active mode) through a very simple combination mapping of the opening and closing of two control switches. This ingenious design not only gives the self-powered communication system a great degree of freedom in micro-resource scheduling, enabling the system to selectively allocate extremely limited collected energy "on demand" based on the rapidly changing channel quality status (implicitly represented by mobile user location information) and dynamically fluctuating battery power status (represented by currently available energy), but also fundamentally overcomes the fatal technical defects of traditional single pure passive reflectors, which have limited performance, or pure active reflectors, which have excessive power consumption and cannot survive long-term in self-powered scenarios. Ultimately, while ensuring an absolute balance between system energy expenditure and revenue, it achieves global optimization of communication network transmission throughput performance and energy utilization efficiency.

[0195] Through steps 100 to 300 above, a closed-loop parameter optimization mechanism that is "channel state information-free and energy-adaptive" is constructed. The overall beneficial effects of the scheme in this application are as follows: it cleverly uses mobile user location parameters to replace the resource-intensive channel state information estimation, fundamentally eliminating massive pilot overhead and estimation errors in dynamic environments, greatly enhancing the deployability of the intelligent reflector communication system in actual mobile scenarios; at the same time, it transforms the stochastic energy harvesting process into precise cross-location scheduling parameters, and realizes the extreme allocation of micro-hardware resources under limited power constraints by relying on a deep reinforcement learning model. This joint and precise control of the system's idle, passive, and active modes and their continuous / discrete transmission parameters successfully overcomes the dual defects of limited performance of traditional pure passive structures and excessive power consumption of pure active structures. Ultimately, while ensuring absolute safety of energy income and expenditure, it maximizes the energy utilization efficiency in limited environments and significantly improves the global average throughput performance of the wireless communication system.

[0196] This application also provides an electronic device, including: At least one memory; At least one processor; At least one program; The program is stored in memory, and the processor executes at least one program to implement the parameter optimization method for the hybrid self-powered reflective surface communication system based on mobile user location information described above in this application. The electronic device can be any smart terminal, including mobile phones, tablets, personal digital assistants (PDAs), and in-vehicle computers.

[0197] Please see Figure 5 , Figure 5 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes: The processor 501 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application. The memory 502 can be implemented in the form of ROM (Read-Only Memory), static storage device, dynamic storage device, or RAM (Random Access Memory). The memory 502 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 502 and is called and executed by the processor 501 to execute the parameter optimization method of the hybrid self-powered reflective surface communication system based on mobile user location information according to the embodiments of this application. The input / output interface 503 is used to implement information input and output; The communication interface 504 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 505 transmits information between various components of the device (e.g., processor 501, memory 502, input / output interface 503, and communication interface 504); The processor 501, memory 502, input / output interface 503, and communication interface 504 are connected to each other within the device via bus 505.

[0198] This application embodiment also provides a storage medium, which is a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the parameter optimization method of the hybrid self-powered reflective surface communication system based on mobile user location information described above.

[0199] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. In some embodiments, memory may optionally include memory remotely located relative to the processor, which can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0200] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A parameter optimization method for a hybrid self-powered reflector communication system based on mobile user location information, characterized in that, The communication system includes a transmitter, a hybrid smart reflector equipped with energy harvesting circuitry, and a mobile user. Each reflector element of the hybrid smart reflector can switch between idle mode, passive mode, and active mode. The method includes: Obtain the current location parameters of the mobile user at the current time step, and obtain the current available energy of the hybrid smart reflector at the current time step; The current position parameters and the current available energy input transmission parameters are optimized by data processing to obtain the target operating mode, target phase shift and target amplification factor; In the current time step, the operating mode of each reflection unit in the hybrid intelligent reflective surface is adjusted according to the target operating mode, the phase shift of each reflection unit is adjusted according to the target phase shift, and the amplification factor of the reflection unit operating in the active mode is adjusted according to the target amplification factor.

2. The parameter optimization method for a hybrid self-powered reflector communication system based on mobile user location information according to claim 1, characterized in that, Each of the aforementioned reflective units includes a phase-shifting circuit, a reflective amplifier, a first switch, and a second switch; adjusting the operating mode of each reflective unit in the hybrid intelligent reflective surface according to the target operating mode includes: When the target operating mode is idle mode, the first switch is turned off to shut down the corresponding reflection unit; When the target operating mode is passive mode, the first switch is controlled to close and the second switch is controlled to open, thereby activating the phase shifting circuit; When the target operating mode is active mode, both the first switch and the second switch are closed to simultaneously activate the phase-shifting circuit and the reflective amplifier.

3. The parameter optimization method for a hybrid self-powered reflector communication system based on mobile user location information according to claim 1, characterized in that, The transmission parameter optimization model is derived from a pre-constructed system optimization problem. The process of constructing the system optimization problem includes: The optimization objective is to maximize the average throughput of information received by the mobile user within the mobile area. The operating mode parameters, phase shift parameters, and amplification coefficient parameters of each of the reflection units of the hybrid intelligent reflective surface are used as optimization variables. Based on the operating mode parameters, phase shift parameters, and amplification factor parameters of each of the aforementioned reflective units, service quality constraints and energy causality constraints are determined. Based on the optimization objective, the optimization variables, the service quality constraints, and the energy causality constraints, the system optimization problem is obtained.

4. The parameter optimization method for a hybrid self-powered reflector communication system based on mobile user location information according to claim 3, characterized in that, The optimization objective of maximizing the average throughput of information received by the mobile user within the mobile area includes: The average bit error rate of information reception by the mobile user at various locations within the mobile area is obtained. The decoding accuracy is obtained by subtracting the average bit error rate of the received information from the first value. Based on the decoding accuracy, multiply by the transmission rate of the information sent by the transmitter and the dwell time of the mobile user at each location to obtain the average number of bits that the mobile user can correctly decode at each location. The mathematical expectation of the average number of bits corresponding to all positions within the mobile area is obtained to obtain the average throughput expression; Maximizing the average throughput expression is taken as the optimization objective.

5. The parameter optimization method for a hybrid self-powered reflector communication system based on mobile user location information according to claim 3, characterized in that, The process of constructing the transmission parameter optimization model includes: The state space is obtained based on the location parameters of the mobile user and the available energy parameters of the hybrid smart reflector. Based on the working mode parameters, phase shift parameters, and amplification coefficient parameters of each of the reflection units in the hybrid intelligent reflective surface, the action space is obtained; Based on the service quality constraints and energy causality constraints in the system optimization problem, a reward function model is constructed to maximize the effective throughput of the communication system. Obtain the initial action parameters of the actor network and the initial comment parameters of the commenter network; An initial transmission parameter optimization model is generated based on the action space, the state space, the reward function model, the initial action parameters, and the initial comment parameters. The initial transmission parameter optimization model is then trained multiple times in a simulation environment to obtain the transmission parameter optimization model.

6. The parameter optimization method for a hybrid self-powered reflector communication system based on mobile user location information according to claim 5, characterized in that, The reward function model is constructed based on the service quality constraints and energy causality constraints in the system optimization problem, including: Based on the transmission symbols, cascaded channel model, and additive noise in the hybrid self-powered reflector communication system, the analog received symbols corresponding to each of the reflector units in each of the operating modes are obtained. The average symbol error rate is obtained based on the number of errors in the analog received symbols that were not correctly decoded and the total number of transmitted symbols. The analog bit error rate is obtained by dividing the average symbol error rate by the logarithm of the modulation order. The simulated effective throughput model is obtained by subtracting the simulated bit error rate from the first value and then multiplying it by the transmission rate and dwell time. When the simulated effective throughput model is greater than the preset threshold parameter, and the total energy consumed by the reflection unit in the passive mode and the active mode is not greater than the available energy parameter in the corresponding working mode, the reward function model is obtained based on the simulated effective throughput model. When the simulated effective throughput model is not greater than a preset threshold parameter, or the total energy consumed is greater than the available energy parameter in the corresponding working mode, the reward function model is obtained based on zero.

7. The parameter optimization method for a hybrid self-powered reflector communication system based on mobile user location information according to claim 5, characterized in that, The step of training the initial transmission parameter optimization model in a simulated environment for multiple rounds to obtain the transmission parameter optimization model includes: In the current training time step of each training round, the current training position parameters and the current available training energy are input into the actor network to obtain the training action distribution, and the current training action combination is sampled from the training action distribution; The current training action combination is executed in the simulation environment, and the current immediate reward is obtained based on the reward function model, as well as the updated training state for the next training time step is obtained. The current training position parameters, the current available training energy, the current training action combination, the current instant reward, and the updated training status are stored as training experience information in the experience replay pool. The initial action parameters and the initial comment parameters are iteratively updated using the batch training experience information sampled in the experience replay pool, and the updated actor network is used as the transmission parameter optimization model.

8. The parameter optimization method for a hybrid self-powered reflector communication system based on mobile user location information according to claim 7, characterized in that, The step of iteratively updating the initial action parameters and the initial comment parameters using batch training experience information sampled from the experience replay pool includes: The state value estimate corresponding to the training experience information is obtained using the commentator network. Based on the current immediate reward and the estimated state value, a dominance function model is calculated, which is used to characterize the degree to which the current training action combination is better than the average action. The importance weight of the updated policy is obtained based on the probability ratio between the updated and unupdated actor network policies. The importance weights of the update strategy are numerically truncated based on a preset truncation range parameter to obtain truncated importance weights. The initial action parameters are updated by gradient based on the product of the truncation importance weight and the advantage function model. The initial comment parameters are updated using a gradient based on the mean square error between the target state value and the estimated state value.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the parameter optimization method for the hybrid self-powered reflective surface communication system based on mobile user location information as described in any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the parameter optimization method for the hybrid self-powered reflective surface communication system based on mobile user location information as described in any one of claims 1 to 8.

Images