Push-broom radiometer antenna pattern optimization method based on reinforcement learning

Abstract

The invention provides a push-broom radiometer antenna pattern optimization method based on reinforcement learning. The push-broom radiometer antenna pattern optimization method comprises the steps of S1, initializing an intelligent agent and an environment; s2, initializing network input and network output; s3, selecting an antenna pattern evaluation index, wherein the antenna pattern evaluation index comprises main beam efficiency, main beam width, side lobe height and a difference between the main beam efficiency, the main beam width and the side lobe height and an ideal antenna pattern; s4, constructing a deep neural network: designing the deep neural network into a structure combining a convolutional neural network and a deep neural network; s5, training a deep neural network; and step S6, obtaining an antenna pattern after network optimization. According to the optimization method, the sidelobe level can be lowered, and the main beam efficiency is improved, so that the system detection precision is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of satellite communications, relates to antenna patterns, and specifically relates to a method for optimizing the antenna patterns of a push-broom radiometer based on reinforcement learning. Background Art

[0002] In view of the high main beam efficiency and profile accuracy compensation requirements of the digital beamforming push-scan radiometer, the traditional single feed source cannot meet the illumination requirements, and thus cannot meet the system performance requirements for the antenna pattern. Instead, a dense feed source array composed of a large number of smaller feed sources can be used to form an antenna beam with low side lobes, high efficiency, and flexible pointing that meets the system requirements by optimizing the amplitude and phase weighting coefficients.

[0003] Traditional antenna pattern optimization methods, such as those based on gradient algorithms and genetic algorithms, have problems such as excessive computational complexity and difficulty in selecting optimization parameters, which results in higher side lobes in the antenna pattern and affects the detection accuracy of the system. Summary of the invention

[0004] In view of the shortcomings of the prior art, the purpose of the present invention is to provide a push-broom radiometer antenna pattern optimization method based on reinforcement learning, so as to solve the technical problem that the system detection accuracy of the optimization method in the prior art needs to be further improved.

[0005] In order to solve the above technical problems, the present invention adopts the following technical solutions to achieve the above problems:

[0006] A method for optimizing antenna pattern of a push-broom radiometer based on reinforcement learning, the method comprising the following steps:

[0007] Step S1, initialize the agent and environment.

[0008] Step S2, initializing network input and network output.

[0009] Step S3, selecting antenna pattern evaluation index:

[0010] The antenna pattern evaluation indexes include main beam efficiency, main beam width, side lobe height and the difference from the ideal antenna pattern.

[0011] Step S4, construct a deep neural network:

[0012] The deep neural network is designed as a structure combining a convolutional neural network and a deep neural network.

[0013] Step S5, training the deep neural network.

[0014] Step S6, obtaining the antenna pattern after network optimization.

[0015] Compared with the prior art, the present invention has the following technical effects:

[0016] (I) The optimization method of the present invention can reduce the sidelobe level and improve the main beam efficiency, thereby improving the system detection accuracy.

[0017] (II) The antenna pattern evaluation indicators of the present invention include main beam efficiency, main beam width, side lobe height, and ideal antenna pattern. Different evaluation indicators correspond to different optimization weight coefficients. The weight coefficients can be adjusted in real time on orbit to meet different application requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 A schematic diagram of a digital beamforming push-scan radiometer provided by an embodiment of the present invention.

[0019] Figure 2 A flow chart of a method for optimizing the antenna pattern of a push-broom radiometer based on reinforcement learning provided in an embodiment of the present invention.

[0020] Figure 3 A schematic diagram of a deep neural network structure provided by an embodiment of the present invention.

[0021] Figure 4 A deep neural network training flow chart provided by an embodiment of the present invention.

[0022] Figure 5 A 48-unit dense feed array is used for testing provided in the application example of the present invention.

[0023] Figure 6 A single feed directional diagram is provided for an application example of the present invention.

[0024] Figure 7 The optimized antenna pattern provided for the application example of the present invention.

[0025] The specific contents of the present invention are further explained in detail below in conjunction with embodiments. DETAILED DESCRIPTION

[0026] It should be noted that, unless otherwise specified, all devices, algorithms and networks in the present invention adopt devices, algorithms and networks known in the prior art.

[0027] In the present invention, the push-broom radiometer is a digital beamforming push-broom radiometer. Figure 1 A schematic diagram of a digital beamforming push-scan radiometer is shown. For ease of explanation, only the parts related to the present invention are shown, which are described in detail as follows:

[0028] The digital beamforming push-scan radiometer includes a ring-focus reflector, a dense feed array, a receiving channel array, a data processor, etc. The ring-focus reflector and the dense feed array provide the initial single feed pattern, and the optimized amplitude and phase weight parameters are injected into the system through the data processor to form the final optimized antenna pattern.

[0029] Specific embodiments of the present invention are given below. It should be noted that the present invention is not limited to the following specific embodiments, and all equivalent changes made on the basis of the technical solution of this application fall within the protection scope of the present invention.

[0030] Example:

[0031] This embodiment provides a method for optimizing the antenna pattern of a push-broom radiometer based on reinforcement learning. Figure 2 As shown, the method comprises the following steps:

[0032] Step S1, initialize the agent and environment:

[0033] In step S1, the intelligent agent is a feed array of a push-scan radiometer.

[0034] In step S1, the environment is a process of obtaining an optimized system array factor from the antenna radiation pattern of each feed in the feed array.

[0035] In step S1, the system array factor is the antenna pattern of the system feed array.

[0036] Step S2, initialize network input and network output:

[0037] In step S2, the network input is the antenna pattern of each feed source and the initialized amplitude and phase weights.

[0038] In step S2, the network output is the amplitude and phase weight parameters required to be adjusted for all channels, and the network output serves as the basis for adjusting the antenna pattern.

[0039] Step S3, selecting antenna pattern evaluation index:

[0040] Antenna pattern evaluation indicators include main beam efficiency, main beam width, side lobe height, and the difference from the ideal antenna pattern.

[0041] In step S3, the optimized antenna pattern is obtained according to the amplitude and phase weight parameters that need to be adjusted for all channels output by the network, and the antenna pattern is evaluated through indicators to see whether it meets the requirements.

[0042] In step S3, there are multiple indicators that can be used to measure the quality of the antenna pattern, such as main beam efficiency, main beam width, side lobe height, and the difference from the ideal antenna pattern. Different evaluation indicators correspond to different amplitude and phase weight parameters, and the satellite can adjust the amplitude and phase weight parameters in real time in orbit to meet different application requirements.

[0043] Main beam efficiency: In an antenna, the beam range (or beam solid angle) is composed of the main lobe range (or solid angle) plus the side lobe range (or solid angle). The (main) beam efficiency is the ratio of the power radiated (or received) by the (main) beam to the total power radiated (or received) by the (total) beam.

[0044] Main beam width: The width of the main lobe defined by a 3dB decrease near the maximum value of the main lobe.

[0045] Sidelobe height: The sidelobe level height of an antenna refers to the maximum sidelobe, usually the ratio of the first sidelobe to the maximum value of the main lobe.

[0046] Difference from ideal antenna pattern: Determined by comparing the input antenna pattern with the antenna pattern after output parameter adjustment, and can be determined using evaluation indicators such as RMSE.

[0047] Step S4, construct a deep neural network:

[0048] The deep neural network is designed as a structure that combines convolutional neural network and deep neural network.

[0049] In step S4, Figure 3 As shown in the figure, the convolutional neural network is first used to extract the features of the pixel matrix, and then the deep neural network is used to fit the nonlinear mapping from the input state to the output action value. The antenna pattern and the corresponding amplitude and phase weight parameters of the network input are respectively extracted through multiple layers of convolutional layers, expanded into a fully connected layer, and then passed through multiple layers of fully connected neural networks, and finally the amplitude and phase weight parameters that need to be adjusted for each channel are output.

[0050] Step S5, training the deep neural network:

[0051] In step S5, the antenna pattern of each feed source and the initialized amplitude and phase weights are input into the deep neural network as state t0. If this state does not exist in the experience pool, K parameter adjustment directions are randomly selected; if this state appears, K directions with the maximum expected value are selected. After inputting state t0, the network outputs action a1, and all channels adjust parameters based on action a1 to obtain the antenna pattern corresponding to state t1, calculate the feedback parameter r1, and input it into the network. Store state t0, action a1, state t1, and feedback parameter r1 in the experience pool, calculate the loss function value, use the Adam optimization algorithm to optimize the deep neural network, and update the network parameters, such as Figure 4 shown.

[0052] In step S5, in the deep neural network, the same deep neural network is used to calculate the next target expected value and estimate the current expected value, where the expected value refers to the expectation of the antenna pattern evaluation index adjusted from the current feed array amplitude and phase parameters to the next different amplitude and phase parameters. In the simulation, it will cause the network oscillation to be difficult to converge, so the target network is introduced to overcome this unstable defect. The target network does not interact with the environment, nor is it updated at every step, but is updated at a certain number of steps according to the network design, that is, the current network parameters are directly assigned to the target network.

[0053] In step S5, since the training data are required to be independent and identically distributed, this condition is not met in antenna pattern optimization. Experience replay is used to solve this problem. Experience replay means that during the network and training process, the existing input and corresponding output are stored in the experience pool in the form of an array. Each training will randomly sample a set small batch of data from the experience pool for auxiliary training, so as to achieve the purpose of eliminating the correlation between samples to a certain extent.

[0054] Step S6, obtaining the antenna pattern after network optimization:

[0055] In step S6, when the antenna pattern obtained by the amplitude and phase adjustment parameters output by the deep neural network meets the antenna pattern evaluation index, it is the final optimized antenna pattern.

[0056] Application examples:

[0057] This application example provides a method for optimizing the antenna pattern of a push-broom radiometer based on reinforcement learning in the above embodiment. Figure 2 As shown, the method comprises the following steps:

[0058] In this application example, the feed array uses Figure 5 The 48-element densely fed array is shown.

[0059] In this application example, an antenna pattern test is performed on each feed of the system feed array in a darkroom to obtain antenna pattern data.

[0060] In this application example, the main beam efficiency is selected as the antenna pattern evaluation index, and the main beam efficiency is set to be greater than 99%, which means that the index is met.

[0061] In this application example, the antenna pattern of each feed source and the initialized amplitude and phase weights are input into the deep neural network.

[0062] In this application example, the antenna pattern obtained by the amplitude and phase adjustment parameters output by the deep neural network guides the adjustment of network parameters.

[0063] In this application example, the training is completed when the main beam efficiency of the optimized antenna pattern is 99%.

[0064] The main beam efficiency of the antenna after optimization based on deep neural network is 99.1%. Figure 6 As shown. Before optimization, the main beam efficiency of the single feed source is 81.2%, as shown Figure 7 The push-broom radiometer antenna pattern optimization method based on reinforcement learning can significantly improve the main beam efficiency of the antenna.

Claims

1. A method for optimizing the antenna pattern of a push-broom radiometer based on reinforcement learning, characterized in that: The method comprises the following steps: Step S1, initialize the agent and environment; Step S2, initializing network input and network output; Step S3, selecting antenna pattern evaluation index: The antenna pattern evaluation index includes main beam efficiency, main beam width, side lobe height and difference from the ideal antenna pattern; Step S4, construct a deep neural network: The deep neural network is designed as a structure combining a convolutional neural network and a deep neural network; Step S5, training a deep neural network; Step S6, obtaining the antenna pattern after network optimization.

2. The method for optimizing the antenna pattern of a push-broom radiometer based on reinforcement learning according to claim 1, characterized in that: In step S1, the intelligent agent is a feed array of a push-scan radiometer.

3. The method for optimizing the antenna pattern of a push-broom radiometer based on reinforcement learning according to claim 1, characterized in that: In step S1, the environment is a process of obtaining an optimized system array factor from the antenna radiation pattern of each feed in the feed array.

4. The method for optimizing the antenna pattern of a push-broom radiometer based on reinforcement learning according to claim 3, characterized in that: In step S1, the system array factor is the antenna pattern of the system feed array.

5. The method for optimizing the antenna pattern of a push-broom radiometer based on reinforcement learning according to claim 1, wherein: In step S2, the network input is the antenna pattern of each feed source and the initialized amplitude and phase weights.

6. The method for optimizing the antenna pattern of a push-broom radiometer based on reinforcement learning according to claim 1, wherein: In step S2, the network output is the amplitude and phase weight parameters required to be adjusted for all channels, and the network output is used as the basis for adjusting the antenna pattern.

7. The method for optimizing the antenna pattern of a push-broom radiometer based on reinforcement learning according to claim 1, wherein: In step S4, a convolutional neural network is first used to extract features of the pixel matrix, and then a deep neural network is used to fit a nonlinear mapping from the input state to the output action value; the antenna pattern input to the network and the corresponding amplitude and phase weight parameters are respectively extracted through multiple layers of convolutional layers, expanded into a fully connected layer, and then passed through multiple layers of fully connected neural networks, and finally the amplitude and phase weight parameters that need to be adjusted for each channel are output.

Images