A Fast Phase Shift Optimization Method for Intelligent Reflective Surface-Assisted UAV Line Inspection System

By optimizing the IRS phase shift through transfer learning and model-free reinforcement learning, the problem of rapid phase shift optimization in UAV line-following systems is solved, thereby improving spectral efficiency, shortening training time, and adapting to dynamic channel changes.

CN117336735BActive Publication Date: 2026-11-17NANJING UNIV OF AERONAUTICS & ASTRONAUTICS
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202311269005.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-09-28
Publication Date
2026-11-17
Estimated Expiration
2043-09-28

AI Technical Summary

Technical Problem

Existing technologies struggle to quickly optimize the phase shift of intelligent reflective surfaces in UAV line-following systems, especially in dynamic channel environments where maximizing spectral efficiency is difficult. Furthermore, existing methods often involve long training times or are unable to adapt to channel variations.

Method used

By employing transfer learning techniques, an IRS-assisted UAV line-following system model is constructed, a Markov decision process is designed, and a model-free reinforcement learning method is used to train the agent. Combined with a two-layer deep Q-network and a transfer knowledge base, the IRS phase shift is quickly optimized to improve spectral efficiency.

Benefits of technology

It effectively improves communication between UAVs and ground nodes in a short period of time, increases spectrum efficiency, reduces training time, and avoids negative migration effects, which is superior to traditional methods.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117336735B_ABST
    Figure CN117336735B_ABST
Patent Text Reader

Abstract

This invention discloses a fast phase shift optimization method for intelligent reflective surface-assisted UAV line inspection systems, comprising the following steps: (1) constructing an IRS-assisted UAV line inspection system model and its dynamic channel model; (2) constructing system performance evaluation indicators, and establishing an optimization problem based on the indicators to maximize spectral efficiency by adjusting the IRS phase shift; (3) modeling the optimization problem as a Markov decision process, and training an agent capable of autonomously adjusting the IRS phase shift using a model-free reinforcement learning method; (4) using transfer learning to accelerate the training process, including constructing a transfer knowledge base, task matching, and hot start. This invention utilizes transfer learning technology to design a fast IRS phase shift optimization algorithm for IRS-assisted UAV line inspection scenarios in factories, maximizing the spectral efficiency of the entire communication system under limited UAV transmission power.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention belongs to the field of intelligent reflective surface-assisted communication technology, specifically relating to a fast phase shift optimization method for intelligent reflective surface-assisted UAV line inspection systems. Background Technology

[0002] Intelligent reflecting surfaces (IRS) are made of subwavelength artificial materials and can adjust their reflection of electromagnetic waves by changing their reflection coefficient. Deploying IRS in communication systems and controlling their reflection allows for the reconfiguration of the radio propagation environment, significantly improving the performance of wireless communication networks. IRS possesses advantages such as adaptability and reconfigurability, making them highly suitable for current industrial production needs. Numerous studies have explored the application of IRS in the Industrial Internet of Things (IIoT), such as deploying IRS at 5G cell boundaries to reduce inter-cell interference and improving the throughput of vehicle-to-everything (V2X) communication systems by optimizing IRS phase shift. A series of important works have innovatively studied phase shift optimization methods for IRS-assisted communication, with research directions mainly divided into three areas: 1) using mathematical methods based on convex optimization to correct IRS phase shift; 2) using reinforcement learning-based agents to dynamically adjust IRS phase shift; and 3) improving the communication rate of users assisted by IRS.

[0003] It is worth noting that many communication devices assisted by IRS have highly limited usage time, especially considering their actual size, weight, and battery capacity. For example, drones currently on the market, such as DJI drones, have a flight time of approximately 530 minutes. Therefore, how to configure the IRS phase shift matrix in the shortest possible time is a particularly urgent and critical issue. Despite extensive theoretical research, the practical deployment of IRS in the Industrial Internet still faces numerous difficulties. First, the IRS itself is a passive device and cannot autonomously transmit or receive pilot signals to acquire channel information; how to acquire channel information is one of the key challenges facing IRS-assisted communication. Second, the wireless channels faced by the IRS are highly dynamic and easily affected by various factors, including multipath effects, weather conditions, and external electromagnetic interference. To adapt to the dynamic changes in the wireless channel, the IRS phase shift needs to be optimized in real time to ensure high-speed communication between users; how to efficiently update the phase shift after acquiring channel information is another key challenge facing IRS-assisted communication. Based on the above problems, this invention takes a specific application case of an intelligent reflective surface-assisted drone line-following system as the research object. Specifically, the development of this system has the potential to improve communication efficiency. By introducing transfer learning, the efficiency of IRS phase shift configuration can be effectively improved, and the communication between UAVs and ground nodes can be improved in a short period of time.

[0004] Existing work on IRS phase shift correction based on convex optimization is mainly applicable to static channel modeling scenarios and does not consider channel changes caused by external interference or user movement. Work on IRS phase shift adjustment based on reinforcement learning often requires long training times and can only use models that have already been trained offline, making it difficult to cope with potential changes in the scenario, such as fine-tuning of user movement trajectories. How to achieve rapid optimization of IRS phase shift is an urgent problem to be solved. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to address the shortcomings of the prior art by providing a fast phase shift optimization method for intelligent reflective surface-assisted UAV line inspection systems. By utilizing transfer learning technology, a fast phase shift optimization algorithm for IRS-assisted UAV line inspection scenarios in factories is designed to maximize the spectral efficiency of the entire communication system under the condition of limited UAV transmission power.

[0006] To achieve the above-mentioned technical objectives, the technical solution adopted by the present invention is as follows:

[0007] A fast phase-shift optimization method for intelligent reflective surface-assisted UAV line-following systems includes the following steps:

[0008] (1) Construct an IRS-assisted UAV line-following system model and its dynamic channel model;

[0009] (2) Construct system performance evaluation indicators, and based on the indicators, establish an optimization problem to maximize spectral efficiency by adjusting the IRS phase shift;

[0010] (3) The optimization problem is modeled as a Markov decision process, and a model-free reinforcement learning method is used to train an agent that can autonomously adjust the IRS phase shift.

[0011] (4) Use transfer learning methods to accelerate the training process, including building a transfer knowledge base, task matching, and hot start.

[0012] To optimize the above technical solution, the specific measures also include:

[0013] The IRS-assisted UAV line inspection system model described in step (1) above includes a ground data center, an IRS with M array units, and a UAV group containing K UAVs; wherein all UAVs maintain a constant altitude and take pictures of equipment in the factory, the ground data center maintains real-time communication with the UAV group and receives the pictures it takes, and the IRS improves the channel conditions between the UAVs and the data center.

[0014] The dynamic channel model described in step (1) above represents the channel between the k-th UAV and the data center as follows: in This indicates the channel between the drone and the data center. h represents the channel between the drone and the IRS. ID Let φ represent the channel between the IRS and the data center, t represent the phase shift vector of the IRS, and t represent time. The channel between the entire UAV ensemble and the data center is represented as H(t) = H UD (t)+H UI (t)φ(t)H ID H(t), where H(t) = [h1(t), h2(t), ..., h K (t)],H UD H UI and H ID All are represented using the Ricean channel model;

[0015] The dynamic channel model describes channel changes through time slots. Each time slot contains multiple sub-time slots. In each sub-time slot, the UAV sends a data packet containing pilot and image content to the data center. The data center receives the pilot and unpacks the data. The line-of-sight channel component changes once per time slot, and the UAV position change is recorded once per time slot. The non-line-of-sight channel component changes once per sub-time slot.

[0016] The system performance evaluation index mentioned in step (2) above is: SNR k (t) represents the signal-to-noise ratio from the k-th drone to the data center;

[0017] The optimization problem is:

[0018] θ represents the phase shift matrix of the IRS. The phase shift coefficient of the IRS is represented by , and M represents the number of IRS array cells.

[0019] Step (3) above models the optimization problem as a Markov decision process and represents it using a quintuple (S, A, R, P, γ). At each time step, the agent observes the state s. t Execute action a t And receive rewards R from feedback in the environment. t Where S, A, and R represent the sets of states, actions, and rewards, respectively, P represents the state transition equation, and γ represents the discount factor.

[0020] The aforementioned agent is an intelligent controller capable of autonomously adjusting the IRS phase shift. It is connected to the IRS via a wired connection and obtains the optimal IRS control strategy through interaction with the wireless environment. The state of the agent is {θ, H}. ′}, θ represents the phase shift of the IRS, H ′The channel state information of the cascaded channel obtained after channel estimation in the data center is represented; the action selectable by the agent is the phase shift increment of the IRS; the agent selects the action using a greedy algorithm; the reward obtained by the agent after interacting with the environment is: Where η a η represents the current spectral efficiency at the data center. th V represents the minimum spectral efficiency that a data center can tolerate. p This is a penalty item.

[0021] The reinforcement learning described above uses a two-layer deep Q-network (DDQN), with an objective Q-value of:

[0022]

[0023] w t Represents the current network parameters. This represents the target network parameter values.

[0024] The above-mentioned construction of the transfer knowledge base is as follows: Before the UAV performs the line inspection task in the factory, it usually plans multiple reference routes in advance. Then, the UAV is controlled to fly strictly according to these reference routes. At the same time, the intelligent controller connected to the IRS interacts with the wireless environment and learns the configuration strategy of the IRS. After training according to the reinforcement learning method described in step (3), the intelligent agent model parameters are saved. The phase shift configuration under the reference route is the source task in the transfer learning.

[0025] The task matching described above specifically involves the intelligent controller searching for the source task most similar to the current target task in the transfer knowledge base. The similarity between the source and target tasks is measured using the similarity of the UAV flight trajectories. The Longest Common Subsequence (LCSS) algorithm is used to quantitatively analyze the similarity between UAV trajectories. This algorithm first compares the distances between points in two trajectories; if the distance between points is less than a set threshold, the two points are considered similar. Next, the number of similar points and the number of points in each of the two trajectories are counted. The similarity expression for trajectory A and trajectory B is as follows:

[0026]

[0027]

[0028] `length(LCSS(A,B))` represents the length of the similar portion between path A and path B, and `length(A)` represents the length of the similar portion between path A and path B.

[0029] `length(B)` represents the length of path A, and `length(B)` represents the length of path B.

[0030] The aforementioned hot start specifically involves: after finding the source task with the highest similarity to the target task during the task matching phase, the intelligent controller sets the model parameters of the source task to the initial values ​​of the current target task's model parameters. Simultaneously, it reduces the value of parameter ε in the greedy algorithm, making the agent more inclined to utilize existing experience when selecting actions, thereby accelerating the algorithm's convergence process. The corresponding new greedy algorithm expression is as follows:

[0031]

[0032] Where A t π(a|s) represents the exploration rate decay coefficient of the greedy algorithm; π(a|s) represents the action selection strategy executed by the intelligent controller; argmaxQ π (s,a) represents directly selecting the action with the largest Q value; ε represents the exploration rate of the greedy algorithm.

[0033] The present invention has the following beneficial effects:

[0034] This invention proposes a fast phase shift optimization method for intelligent reflective surface-assisted UAV line inspection systems. The intelligent reflective surface can improve the wireless channel conditions of the UAV by changing its own phase shift to achieve channel reconstruction. In order to improve the communication between the UAV and the ground data center in a short time, i.e., to quickly realize the phase shift configuration of the intelligent reflective surface, transfer learning technology is adopted, and the problem of maximizing the spectral efficiency between the UAV and the ground node is proposed. Under the constraint of the UAV's transmit power, the phase shift optimization of the intelligent reflective surface is designed. Considering the high dynamics of the UAV trajectory and the wireless channel, a model-free reinforcement learning method based on DDQN network is designed to solve the phase shift optimization problem. The analysis and simulation results show that the proposed scheme is superior to the current technology in terms of spectral efficiency and algorithm convergence time.

[0035] This invention applies model-free reinforcement learning to an IRS-assisted UAV line-following system and accelerates the algorithm's convergence speed through transfer learning. To minimize the potential negative effects of transfer learning, this invention employs the longest common subsequence method to calculate the similarity between the flight trajectories of different UAVs and uses this to quantitatively analyze the similarity between the source and target tasks, selecting the source task most similar to the current target task during transfer learning. Simulation results show that the proposed transfer learning scheme effectively accelerates the algorithm's convergence process while effectively avoiding the harm of "negative transfer." Attached Figure Description

[0036] Figure 1 This is a flowchart of the method of the present invention;

[0037] Figure 2 This is a schematic diagram of the system model involved in an embodiment of the present invention;

[0038] Figure 3 This is a reinforcement learning algorithm design scheme involved in the embodiments of the present invention;

[0039] Figure 4 This is a schematic diagram of the task matching process involved in an embodiment of the present invention;

[0040] Figure 5 This invention provides a performance comparison of the transfer learning method before and after its implementation, as well as a performance comparison with the classic multi-armed slot machine algorithm.

[0041] Figure 6 This is a comparison of transfer learning performance under different task similarity levels in the embodiments of the present invention. Detailed Implementation

[0042] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0043] Although the steps in this invention are arranged by reference numerals, this is not intended to limit the order of the steps. Unless the order of the steps is explicitly stated or the execution of a step requires other steps as a basis, the relative order of the steps can be adjusted. It is understood that the term "and / or" as used herein refers to and covers any and all possible combinations of one or more of the associated listed items.

[0044] This embodiment is based on Figure 2 As shown in the IRS-assisted UAV line-following system model, Figure 1 The flowchart shown is a flowchart of the present invention. Figure 1 As shown, the present invention provides a fast phase-shift optimization method for a smart reflective surface-assisted UAV line-following system, comprising the following steps:

[0045] (1) Construct an IRS-assisted UAV line-following system model and its dynamic channel model;

[0046] 1) Construct an IRS-assisted UAV line-following system model;

[0047] The system comprises a ground data center, an IRS with M array units, and a drone swarm containing K drones. All drones maintain a constant altitude and image the equipment in the factory. The ground data center needs to maintain real-time communication with the drone swarm and receive its images. Considering the complexity of the factory environment, the communication link between the drones and the data center is likely to be blocked by obstacles. However, a high communication rate is required between the drones and the data center; therefore, deploying an IRS is considered to improve the channel conditions between the drones and the data center. At time t, the signal received by the data center can be represented as Y(t) = H(t)X(t) + N, where H represents the cascaded channel consisting of the drone-data center channel, the drone-IRS channel, and the IRS-data center channel. X represents the signal transmitted by the drone swarm. N represents additive white Gaussian noise.

[0048] 2) Construct a dynamic channel model;

[0049] The channel between the k-th drone and the data center can be represented as: in This indicates the channel between the drone and the data center. h represents the channel between the drone and the IRS. ID This represents the channel between the IRS and the data center, where φ represents the phase shift vector of the IRS.

[0050] The channel for the entire drone formation and data center is represented as H(t) = H UD (t)+H UI (t)φ(t)H ID H(t), where H(t) = [h1(t), h2(t), ..., h K (t)].

[0051] Specifically, H UD H UI and H ID All are represented using the Ricean channel model.

[0052] With H UD For example, each channel model can be represented as Where H UD,LOS H represents the line-of-sight channel. UD,NLOS K represents a non-line-of-sight channel. r This represents the ratio of power attenuation between line-of-sight and non-line-of-sight channels.

[0053] The phase shift vector of the IRS can be represented by a diagonal matrix: in This represents the phase shift coefficient of the IRS.

[0054] Considering the dynamic nature of wireless channels, the concept of "time slots" is introduced to describe channel changes. The specific settings for system time slots are as follows:

[0055] Sub-time slot: In each sub-time slot, the UAV sends a data packet containing pilot signals and image content to the data center. The data center receives the pilot signals and unpacks the data. It is also assumed that the non-line-of-sight channel component changes once per sub-time slot.

[0056] Time slot: Each time slot contains multiple sub-time slots. Assuming that the line-of-sight channel component changes once per time slot, and that the line-of-sight channel component is closely related to the relative positions of the drone, IRS, and data center, it can also be considered as recording the drone's position change once per time slot.

[0057] (2) Construct system performance evaluation indicators, and based on the indicators, establish an optimization problem to maximize spectral efficiency by adjusting the IRS phase shift;

[0058] Assuming there is no mutual interference between drones, the signal-to-noise ratio from the k-th drone to the data center can be expressed as: P S This indicates the drone's transmission power, which is finite (P). S ≤P S-MAX .

[0059] Without loss of generality, spectral efficiency is used as a metric for evaluating system performance.

[0060] The ultimate goal is to maximize spectral efficiency by adjusting the IRS phase shift; therefore, the optimization problem can be expressed in the following form:

[0061]

[0062] θ represents the phase shift matrix of the IRS. The phase shift coefficient of the IRS is represented by , and M represents the number of IRS array cells.

[0063] (3) The optimization problem is modeled as a Markov decision process, and a model-free reinforcement learning method is used to train an agent that can autonomously adjust the IRS phase shift.

[0064] like Figure 3 The diagram shows the design scheme of the reinforcement learning algorithm, which models the above optimization problem as a Markov decision process and represents it using a quintuple (S, A, R, P, γ).

[0065] S, A, and R represent the sets of states, actions, and rewards, respectively. Within each time step, the agent observes state s. t Execute action a t And receive rewards R from feedback in the environment. t .

[0066] P represents the state transition equation, and γ represents the discount factor, which is used to measure the importance of future rewards versus immediate rewards.

[0067] The reinforcement learning framework used in this invention is a Double Deep Q Network (DDQN).

[0068] The specific details of each of the above parts are described below:

[0069] Intelligent Agent: The intelligent agent in this invention is an intelligent controller capable of autonomously adjusting the phase shift of the IRS. This intelligent controller is connected to the IRS via a wired connection and obtains the optimal IRS control strategy by interacting with the wireless environment.

[0070] State: The state of the agent is defined as {θ, H′}. θ represents the phase shift of the IRS, and H′ represents the channel state information of the cascaded channel obtained after channel estimation by the data center.

[0071] Actions: The actions that the agent can choose are the phase shift increments of the IRS. The specific definition of the action set is as follows:

[0072]

[0073]

[0074] This indicates that the IRS maintains the current phase shift. This indicates that the IRS increases or decreases its own phase shift. ±π indicates an accelerated rate of change in the IRS phase shift. This action set is not unique; users can adjust its size according to scenario requirements. After the agent executes the selected action, the IRS phase shift can be expressed as θ(t+1)=θ(t)☉Δθ(t+1), where ☉ represents the Hadamard product, i.e., the multiplication of corresponding elements in the matrix.

[0075] Reward: The reward obtained by an agent after interacting with the environment is defined as follows:

[0076]

[0077] η a η represents the current spectral efficiency at the data center. th This represents the minimum spectral efficiency that the data center can tolerate. When the spectral efficiency falls below this minimum threshold, the data center cannot properly receive images transmitted from drones; therefore, a large penalty term V is added. p This incentivizes the agent to choose a better phase-shift control strategy.

[0078] ε-greedy algorithm: To avoid getting trapped in local optima, the agent uses a greedy algorithm to select actions.

[0079]

[0080] Two-layer deep Q-network: The two-layer deep Q-network is developed based on the classic deep Q-network. By adding a target network, it effectively alleviates the overestimation problem faced by deep Q-networks. The target Q-value of the DDQN network can be expressed as follows:

[0081]

[0082] w t Represents the current network parameters. This represents the target network parameter values.

[0083] (4) Use transfer learning methods to accelerate the training process, including building a transfer knowledge base, task matching, and hot start.

[0084] The entire transfer learning process can be divided into three stages.

[0085] Building a transfer knowledge base: Before performing line inspection tasks in a factory, UAVs typically plan multiple baseline routes in advance. The UAV is then controlled to fly strictly according to these baseline routes. Simultaneously, the intelligent controller connected to the IRS interacts with the wireless environment and learns the IRS configuration strategy. After training using the reinforcement learning method described in step (3), the agent model parameters are saved. The phase shift configurations under these baseline routes are the source tasks in transfer learning.

[0086] Task matching: such as Figure 4The diagram illustrates the task matching process. Although a baseline route is pre-planned, drones often adjust their flight paths temporarily based on unforeseen circumstances during actual route patrols. For example, if a fire breaks out in a factory area, the drone needs to immediately approach the burning equipment and photograph the damage. In this case, the intelligent controller's objective is to reconfigure the IRS phase shift control strategy for this temporarily adjusted route. To achieve high-speed communication between the drone and the ground data center as quickly as possible, the intelligent controller needs to find the source task most similar to the current objective task in the migration knowledge base. The IRS phase shift configuration is closely related to the small-scale attenuation of the channel, while the small-scale attenuation of the line-of-sight channel is closely related to the drone's flight trajectory. Therefore, this invention uses the similarity of the drone's flight trajectory to measure the similarity between the source and target tasks. The Longest Common Subsequence (LCSS) algorithm is used to quantitatively analyze the similarity between drone trajectories. This algorithm first compares the distances between points in two trajectories; if the distance between points is less than a set threshold, the two points are considered similar. Next, the number of these similar points and the number of points in each of the two trajectories are counted. The final similarity expression for trajectory A and trajectory B is as follows:

[0087]

[0088] The LCSS expression is as follows:

[0089]

[0090] `length(LCSS(A,B))` represents the length of the similar portion between path A and path B, and `length(A)` represents the length of the similar portion between path A and path B.

[0091] `length(B)` represents the length of path A, and `length(B)` represents the length of path B.

[0092] Warm start: After finding the source task with the highest similarity to the target task during the task matching phase, the intelligent controller sets the model parameters of the source task to the initial values ​​of the current target task's model parameters. Simultaneously, the value of parameter ε in the greedy algorithm is reduced, making the agent more inclined to utilize existing experience when selecting actions, thereby accelerating the algorithm's convergence process. The new greedy algorithm expression is as follows:

[0093]

[0094] Where A t π(a|s) represents the exploration rate decay coefficient of the greedy algorithm; π(a|s) represents the action selection strategy executed by the intelligent controller, and argmaxQ. π (s,a) represents directly selecting the action with the largest Q value, and ε represents the exploration rate of the greedy algorithm.

[0095] The simulation analysis is as follows:

[0096] Figure 5 The curves showing the system spectral efficiency versus algorithm iteration time under different IRS phase shift optimization algorithms are presented. The similarity between the source and target tasks is 0.5, and assuming a spectral efficiency threshold of 10 bits / Hz, the following three conclusions can be drawn:

[0097] 1) After using transfer learning, the initial value of spectral efficiency was improved by 10%, effectively achieving a "warm start" for the algorithm;

[0098] 2) After using transfer learning, the convergence speed of the algorithm was greatly improved by 60%, effectively accelerating the learning process;

[0099] 3) Regardless of whether transfer learning is used, the reinforcement learning algorithm proposed in this invention is superior to the traditional multi-armed slot machine algorithm because the traditional multi-armed slot machine algorithm lacks interaction with the wireless environment.

[0100] Figure 6 The curves showing the change in system spectral efficiency as a function of algorithm iteration time after applying transfer learning under different task similarity levels are presented. The main focus is on the acceleration effect brought by transfer learning. Figure 6 Only the performance of the algorithm for the first 600 time slots is shown. The similarity between the source and target tasks is 0.5, 0.38, and 0.2, respectively, and the spectral efficiency threshold is assumed to be 10 bits / Hz. From Figure 6 As can be seen, the acceleration effect of transfer learning weakens as task similarity decreases. When task similarity is too low (similarity = 0.2), the potential negative impact of transfer learning becomes apparent, and the algorithm's performance may even be worse than without transfer learning. Therefore, when performing transfer learning, it is necessary to avoid choosing source tasks with low similarity to the target task.

[0101] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

[0102] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A fast phase-shift optimization method for intelligent reflective surface-assisted UAV line-following systems, characterized in that, Including the following steps: (1) Construct an IRS-assisted UAV line-following system model and its dynamic channel model; (2) Construct system performance evaluation indicators, and based on the indicators, establish an optimization problem to maximize spectral efficiency by adjusting the IRS phase shift; (3) The optimization problem is modeled as a Markov decision process, and a model-free reinforcement learning method is used to train an agent that can autonomously adjust the IRS phase shift; (4) Use transfer learning methods to accelerate the training process, including building a transfer knowledge base, task matching, and hot start; The IRS-assisted UAV line inspection system model described in step (1) includes a ground data center, a system with... An IRS of array cells, and a containing A drone swarm consists of drones; all drones maintain a constant altitude and take images of equipment in the factory. A ground data center maintains real-time communication with the drone swarm and receives the images it takes. An IRS improves the channel conditions between the drones and the data center. The dynamic channel model described in step (1) will... The channel between the drone and the data center is represented as: ,in This indicates the channel between the drone and the data center. This indicates the channel between the drone and the IRS. This indicates the channel between the IRS and the data center. The phase shift vector represents the IRS, and t represents time; Represent the channels of the entire drone formation and data center as follows: ,in , , and All are represented using the Ricean channel model; The dynamic channel model describes the channel changes through time slots. Each time slot contains multiple sub-time slots. In each sub-time slot, the UAV sends a data packet containing pilot and image content to the data center. The data center receives the pilot and unpacks the data. The line-of-sight channel component changes once per time slot, and the UAV position change is recorded once per time slot. The non-line-of-sight channel component changes once per sub-time slot. The system performance evaluation index mentioned in step (2) is: ,in For the first The signal-to-noise ratio from the drone to the data center, where K represents the total number of drones; The optimization problem is: ; This represents the phase shift matrix of the IRS. This represents the phase shift coefficient of the IRS. Indicates the number of IRS array cells; The intelligent agent is an intelligent controller capable of autonomously adjusting the IRS phase shift. It is connected to the IRS via a wired connection and obtains the optimal IRS control strategy by interacting with the wireless environment; the state of the intelligent agent is... , The phase shift representing the IRS This represents the channel state information of the cascaded channel obtained after channel estimation by the data center; The agent can choose actions based on the phase shift increment of the IRS; the agent uses a greedy algorithm to select actions; the reward obtained by the agent after interacting with the environment is: ,in This indicates the current spectrum efficiency at the data center. This indicates the minimum spectral efficiency that a data center can tolerate. This is a penalty item; The task matching specifically involves the intelligent controller searching for the source task most similar to the current target task in the transfer knowledge base. The similarity between the source and target tasks is measured using the similarity of the UAV flight trajectories, and the longest common subsequence (LCSS) algorithm is used to quantitatively analyze the similarity between UAV trajectories. This algorithm first compares the distances between points in the two trajectories. With point Distance between Less than the set threshold Two points can then be considered similar. Next, we count the number of similar points and the number of points in each trajectory. The similarity expression between trajectory A and trajectory B is as follows: in Represents the length of the similar portion between path A and path B. This represents the length of path A. This represents the length of path B; The warm start specifically involves: after finding the source task with the highest similarity to the target task during the task matching phase, the intelligent controller sets the model parameters of the source task as the initial values ​​of the current target task's model parameters, while simultaneously reducing the parameters in the greedy algorithm. The value of makes the agent more inclined to use existing experience when choosing actions, thereby accelerating the convergence process of the algorithm. Accordingly, the new greedy algorithm expression is as follows: in This represents the exploration rate decay coefficient of the greedy algorithm; The action selection strategy executed by the intelligent controller; This means directly selecting the action with the highest Q value; This represents the exploration rate of the greedy algorithm.

2. The fast phase-shift optimization method for a smart reflective surface-assisted UAV line-following system according to claim 1, characterized in that, Step (3) Models the optimization problem as a Markov decision process and uses a quintuple. This indicates that within each time step, the agent observes the state. Execute actions And receive rewards from feedback in the environment. ,in, These represent the sets of states, actions, and rewards, respectively. Represent the state transition equation, This represents the discount factor.

3. The fast phase-shift optimization method for a smart reflective surface-assisted UAV line-following system according to claim 1, characterized in that, The reinforcement learning method employs a two-layer deep Q-network (DDQN) with a target Q-value of: Represents the current network parameters. This represents the target network parameter values.

4. The fast phase-shift optimization method for a smart reflective surface-assisted UAV line-following system according to claim 1, characterized in that, The construction of the transfer knowledge base is as follows: Before the UAV performs the line inspection task in the factory, it usually plans multiple reference routes in advance. Then, the UAV is controlled to fly strictly according to these reference routes. At the same time, the intelligent controller connected to the IRS interacts with the wireless environment and learns the configuration strategy of the IRS. After training according to the reinforcement learning method described in step (3), the intelligent agent model parameters are saved. The phase shift configuration under the reference route is the source task in the transfer learning.