Networking radar multi-target three-dimensional imaging resource scheduling method based on deep reinforcement learning
Through deep reinforcement learning, the radar resource allocation is optimized, and the impact of radar node distribution on imaging quality in ISAR three-dimensional imaging technology is solved, achieving the optimal utilization of radar resources and improving target imaging quality.
Patent Information
- Application Number
- CN202411005114.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-25
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2044-07-25
AI Technical Summary
The existing ISAR three-dimensional imaging technology cannot effectively consider the impact of radar node distribution on imaging quality in multi-target scenarios, resulting in unstable imaging quality and the maximization of radar resources.
The networked radar multi-target three-dimensional imaging resource scheduling method is adopted based on deep reinforcement learning. By perceiving target characteristics, analyzing radar node distribution, building resource scheduling models and designing key elements of reinforcement learning, radar resource allocation is optimized to improve imaging quality.
On the premise of ensuring imaging quality, radar resource consumption is reduced, radar resource optimization scheduling is achieved, and the three-dimensional imaging quality and system efficiency of the target are improved.
Smart Images

Figure CN118981014A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of inverse synthetic aperture radar three-dimensional imaging, and in particular to a method for scheduling multi-target three-dimensional imaging resources of a networked radar based on deep reinforcement learning. Background Art
[0002] Traditional ISAR imaging technology can obtain high-quality two-dimensional images of the target, but it has problems such as geometric distortion and difficulty in azimuth calibration. In addition, the images obtained at different time periods and different viewing angles are quite different, and the precise features of the target cannot be extracted. ISAR three-dimensional imaging technology can further obtain the height dimension information perpendicular to the imaging plane, and the three-dimensional structure of the target is unique, and the three-dimensional image is insensitive to changes in the target's posture. Three-dimensional imaging of the target helps to accurately extract target features, such as shape and size, and effective and stable features are very important for distinguishing different types of targets. Therefore, target three-dimensional imaging plays an important role in target classification and recognition.
[0003] The current ISAR 3D imaging technology is mainly divided into five different types of 3D imaging methods: single pulse and difference beam angle measurement ISAR 3D imaging, ISAR 3D imaging based on scattering center correlation, ISAR 3D imaging based on imaging projection, InISAR 3D imaging and array ISAR 3D imaging. Among them, the ISAR 3D imaging method based on imaging projection mainly selects multiple radars to observe the target at different perspectives, and then reconstructs the target's 3D structure based on the acquired target 2D ISAR image and its projection relationship. This method has a large number of scattering points and good imaging quality, and has better performance than other algorithms.
[0004] However, this method requires multiple radars to image the target at different angles, and different radar distributions will inevitably have different effects on the target's three-dimensional imaging quality; and radar resources are often limited. For multi-target three-dimensional imaging scenarios, we need to allocate appropriate radar resources to each target for imaging, and maximize the overall efficiency of the radar system under limited resource conditions. The existing resource scheduling methods for multi-target three-dimensional imaging of networked radars basically do not consider the impact of radar node distribution on imaging quality, so the scheduling results obtained by these methods cannot guarantee the imaging quality of the target. Summary of the invention
[0005] In view of the shortcomings of the prior art, the present invention proposes a method for resource scheduling of multi-target three-dimensional imaging of networked radar based on deep reinforcement learning, comprising the following steps:
[0006] Step 1: Perceive the target features and calculate the required azimuth resolution of the target;
[0007] Step 2: Determine the two-dimensional imaging plane and projection relationship of each radar to the target;
[0008] Step 3: Analyze the impact of radar node distribution on target 3D imaging quality;
[0009] Step 4: Construct a resource scheduling model for multi-target 3D imaging of networked radars;
[0010] Step 5: Design key elements of reinforcement learning;
[0011] Step 6: Solve the resource scheduling model for multi-target three-dimensional imaging of networked radars, and perform three-dimensional imaging of the target based on the scheduling results.
[0012] Further, the first step is specifically:
[0013] Through the radar tracking results of each target in the detection area, the geographical location, movement speed, distance and yaw angle of each target are obtained; then each radar emits a small number of pulses to perform two-dimensional ISAR coarse imaging of the target to obtain the imaging result s kr (f s ,f τ ), where f s is the fast time dimension frequency, f τ is the slow time dimension frequency, thus estimating the azimuth size S of the target kx and azimuthal sparsity K k ;
[0014] First, the 2D ISAR coarse imaging is normalized:
[0015]
[0016] Then, set the threshold T s ,remember According to the linear relationship between the Doppler frequency and the horizontal coordinate of the target scattering point Calculate the target azimuth size; where f d is the Doppler frequency generated by the relative motion between the target scattering point and the radar, v p is the radial velocity of the scattering point, λ is the wavelength of the transmitted signal, ω is the equivalent rotational angular velocity of the target, x p is the horizontal coordinate of the scattering point; the target azimuth size is expressed as:
[0017]
[0018] Then, the ISAR image is normalized as follows:
[0019]
[0020] Set the threshold T h , s″ kr (f τ ) is discretized into a vector s″ kr , then the azimuth sparsity K of the target k is the vector s″ kr Medium greater than T h The number of elements of ;
[0021] Assume the reference size is S x_ref , the reference azimuth resolution is ρ ref , then according to the target azimuth size, the required azimuth resolution of the target is:
[0022]
[0023] Furthermore, the threshold T in the first step s is 0.5, the threshold T h is: 0.2.
[0024] Further, the second step is specifically as follows:
[0025] Determine the radar line of sight direction vector based on the geographical location of the radar and the target Among them, P k , P r are the geographical locations of the target and radar respectively;
[0026] The target movement speed v k Decompose along the radar line of sight and its orthogonal direction to obtain the target radial velocity v p With tangential velocity v t , where the tangential velocity is expressed as:
[0027]
[0028] For the three-dimensional space where the target is located, a three-dimensional rectangular coordinate system (O, X, Y, Z) is established with the target center O as the origin; for the imaging plane, the W axis is established along the radar line of sight, and the U axis is established along the target tangential velocity direction. The unit vectors of the two axes are u u =v t / |v t |, then the radar imaging plane is represented as (O,U,W);
[0029] Next, the projection relationship is derived. Under far-field conditions, the target ISAR image is considered to be the projection of the target's three-dimensional structure on the imaging plane; point P is a point outside the imaging plane, and point Q is the projection point of point P on the imaging plane. Then The two unit vectors perpendicular to the plane are expressed as:
[0030]
[0031] Let the three-dimensional coordinates of point P be (x, y, z) T , the two-dimensional coordinates of point Q are (u,w) T ,but Expressed as Express equation (6) in the form of matrix multiplication:
[0032]
[0033] Then the projection relationship between the three-dimensional coordinates of the scattering point and the two-dimensional coordinates on the imaging plane is constructed:
[0034]
[0035] in, It is the projection matrix from the three-dimensional coordinates of a point in space to the imaging plane.
[0036] Further, the third step is specifically:
[0037] Design a size N x ·N y ·N z The three-dimensional Boolean matrix X is used as the optimization variable, where the matrix value 1 represents the existence of a target scattering point at that point. The strong scattering points in the two-dimensional ISAR image of the target formed by the i-th radar are extracted according to the peak value of the ISAR image intensity and expressed as G i , the projection image of the target on the imaging plane is represented as F i (X), where the projection function F i (·) is obtained according to the projection matrix φ of the imaging plane; the optimization function is constructed with the goal of minimizing the difference between the target 2D ISAR image and the projection image:
[0038]
[0039] Among them, N R Indicates that there are a total of N R The target is imaged by a radar, and λ is a regularization parameter. The three-dimensional scattering point structure of the target can be reconstructed by solving the optimization model.
[0040] Then the influence of radar node distribution on target imaging quality is analyzed;
[0041] A single two-dimensional image cannot reveal the three-dimensional structure of the target. To locate the three-dimensional scattering points of the target, each target must be imaged by at least two radars. Suppose there are two points P1 and P2 in space. Two radars image the target. The projection points of P1 and P2 on the two imaging planes are Q i1 , Qi2 , Q j1 , Q j2 ,set up The normal vector of the imaging plane i is n i ,θ i for With n i The angle between i1 With Q i2 The distance between is expressed as:
[0042]
[0043] When l i ≥ρ i When P1 and P2 can be distinguished on the imaging plane i, ρ i is the two-dimensional resolution of imaging plane i, and its equivalent condition is: If two points can be distinguished on at least one imaging plane, correct reconstruction of the two points can be achieved;
[0044] Therefore, the condition for achieving correct reconstruction of the target is expressed as:
[0045]
[0046] C0 is the target scattering point set, N is the number of radars observing the target; this condition is transformed into only the distribution of radar nodes, expressed as:
[0047]
[0048] where θ ij is the angle between the two radar imaging planes, ρ rec is the target 3D resolution, θ ij Obtained by the angle between the normal vectors of the two imaging planes;
[0049] The normal vector n of the i-th plane and the j-th plane is obtained by cross-multiplying the unit vector basis of the imaging plane. i 、n j , the angle between the normal vectors of the imaging planes is the angle between the imaging planes:
[0050]
[0051] If the two observation radars meet the above conditions, the target scattering point pairs can be distinguished on the imaging plane of at least one of the radars, so the correct reconstruction of the target can be achieved. The influence of the number of observation radars on the imaging quality is further analyzed.
[0052] Furthermore, the third step of further analyzing the influence of the number of observation radars on the imaging quality is specifically as follows:
[0053] Assume that the correct reconstruction result and a certain wrong reconstruction result are X and X', and bring the reconstruction result into the loss function in the optimization model of formula (9). In order to ensure that the final solution of the model is the correct reconstruction result of the target, the following conditions need to be met:
[0054]
[0055] Assume that X contains two scattering points P1 and P2, and X' contains only one of the scattering points. There are N planes that can correctly distinguish the two scattering points P1 and P2. P , then:
[0056]
[0057] The first half of the formula is negative, so N P The larger the value, the greater the probability that the above inequality holds true; for most scattering point pairs, the more radars that image the target, the greater the number of imaging planes N that can correctly distinguish them. P Therefore, to this extent, we believe that the more radars observe a single target, the lower the probability of the target being reconstructed incorrectly and the higher the target imaging quality.
[0058] Further, the fourth step is specifically:
[0059] Construct a resource scheduling model for multi-target 3D imaging of networked radars, assign radars to each target to perform 3D imaging, and minimize the resource consumption of networked radars while ensuring a certain imaging quality.
[0060] Taking the shortest total scheduling time as our objective function, we set the constraint conditions required for target imaging. Suppose there are M radars and N targets to be imaged. We define an M*N dimensional Boolean matrix W as the optimization variable to represent the scheduling scheme. W(m,n)=1 represents that the mth radar images the nth target. Suppose the total time used by the networked radar to complete the multi-target three-dimensional imaging is T, which is solved by the following steps:
[0061] Step 4.1 Extract from the scheme W which targets each radar needs to observe and record them in the radar's target set TSet(m);
[0062] Step 4.2 Prioritize the target based on the target flight speed, the distance between the target and the radar, and the yaw angle. k Perform a comprehensive evaluation, and then calculate the coherent integration time T required for imaging according to the azimuth resolution required for target imaging. k and observation dimension M k ;
[0063] Step 4.3 Each radar allocates sparse pulses to the target in turn according to the priority until imaging of all targets in the target set is completed;
[0064] Step 4.4 Calculate the time T that each radar consumes to complete the 2D ISAR imaging of all targets in its target set. m , and then select the longest time as the total time T used by the networked radar to complete multi-target three-dimensional imaging;
[0065] The above steps are defined as Tconsum(W), and the objective function is expressed as:
[0066] min Tconsum(W) (16)
[0067] Then determine whether solution W meets the target reconstruction condition;
[0068] Assuming that the radar observes all targets, the angle between the normal vectors of the imaging planes of each two radars observing N targets is calculated, and the minimum angle between the imaging planes of the two radars that meets the reconstruction condition is calculated according to formula (12), and they are recorded in The matrix θ of dimension RT and θ pho middle, Represents a total of Radar node combination method;
[0069] Then, for each column of W, the elements are sorted according to θ. RT and θ ρ Perform bitwise AND operations between two elements in the order of dimensional matrix RT, RT(k,n) = 1 means that target n is observed by two radars in the k-th radar pair;
[0070] Then determine whether solution W meets the reconstruction conditions of each goal:
[0071] Satis=[(RT⊙θ RT )>θ pho ] (17)
[0072] If any column in Satis has a radar combination that meets the conditions, that is, It is believed that scheme W can achieve the correct reconstruction of each target; at the same time, in order to reduce the probability of incorrect reconstruction, each target is observed by at least three radars;
[0073] List resource scheduling models:
[0074] min Tconsum(W)
[0075] stn=1,2,...,N
[0076] sum(W(:,n))≥3
[0077] RT=W(i,n)&W(j,n),i,j=1,2,...,M
[0078] Satis=[(RT☉θ RT )>θ pho ]
[0079] sum(Satis(:,n))〉=1 (18).
[0080] Furthermore, the constraints for setting the target imaging requirements in the fourth step are specifically as follows: 1. Each target is imaged by at least two radars that can meet the reconstruction conditions for any scattering point, that is, there are two radars that meet the condition of formula (12); 2. The number of radars observing the target is as large as possible to reduce the probability of erroneous reconstruction, and each target is imaged by at least three radars; 3. Each radar can use a sparse ISAR imaging algorithm based on compressed sensing to observe multiple targets at the same time, so the pulse sequence allocated to each target needs to meet the sparse reconstruction requirements.
[0081] Further, the fifth step is specifically:
[0082] The A2C method and deep reinforcement learning method are used to solve the resource scheduling model and design the key elements of reinforcement learning:
[0083] Step 5.1 State: Each radar node allocation scheme is considered as a State, and the radar node selection scheme for each target is S n , then State is the radar node selection scheme for all targets: S = (S1, S2, ..., S n ), where S n is a Boolean vector of length m;
[0084] Step 5.2 Action: Set the action to target number n + radar node selection scheme S k :a=(n,S k ), that is, each action only changes the radar node selection scheme of one target, among which some radar node selection schemes that do not meet the conditions will be screened out according to the constraints of the resource scheduling model, and the action space will be pruned;
[0085] Step 5.3 State Transition: Directly according to action a=(n,S k ) Change the radar node selection scheme of target n to S k ;
[0086] Step 5.4 State Value: Keep the state value consistent with the objective function of the scheduling model and set it to the negative value of the total time it takes for the networked radar to complete the multi-target three-dimensional imaging task;
[0087] Step 5.5 Reward: Set the reward for each action to the state value of the next state minus the state value of the current state, that is:
[0088] Reward=SV(S n )-SV(S c ) (19)
[0089] Among them, S n represents the next state, S c represents the current state, SV(·) represents the state value; when the discount factor γ is set to a number close to 1, the cumulative return of each round of training is approximately the state value of the last state of the training round minus the state value of the initial state, that is:
[0090]
[0091] Among them, r is the reward value. From formula (20), it can be seen that the state value SV(S0) of the initial state is certain, and the reinforcement learning method is used to solve the state with the maximum state value.
[0092] Further, the sixth step is specifically:
[0093] The specific steps of solving the resource scheduling model using the A2C method are as follows:
[0094] Step 6.1 Initialize the policy network parameters Θ, value network parameters Ω, best_value = -10000
[0095] Step 6.2 for the number of training rounds from 1 to 500:
[0096] Step 6.3 done=False
[0097] Step 6.4 while not done:
[0098] Step 6.5 Based on the strategy π(Θ) and the current state S c Get action a
[0099] Step 6.6 After taking action, get the next state S n and rewards n
[0100] Step 6.7 Calculate TD error and update the two network parameters Θ and Ω
[0101] Step 6.8 if SV(S n )>best_value
[0102] Step 6.9 best_value=SV(S n )
[0103] Step 6.10 end if
[0104] Step 6.11 if SV(S n )==best_value
[0105] Step 6.12 done=True
[0106] Step 6.13 end if
[0107] Step 6.14 S c =S n
[0108] Step 6.15 end while
[0109] Step 6.16 end for
[0110] After the algorithm converges, the optimal radar node scheduling plan can be obtained. Then, each radar will image the targets in its target set by allocating pulses according to priority according to the scheduling plan, and finally reconstruct the three-dimensional image of each target according to the three-dimensional imaging algorithm.
[0111] The advantages of the present invention are as follows: 1. The influence of radar node distribution on the target three-dimensional imaging quality is considered, and a network radar multi-target three-dimensional imaging resource scheduling model is constructed to minimize time resource consumption while ensuring imaging quality. 2. The model is solved by using a deep reinforcement learning method to obtain the optimal solution to the problem. BRIEF DESCRIPTION OF THE DRAWINGS
[0112] Figure 1 A schematic diagram of radar imaging plane coordinate system construction is shown;
[0113] Figure 2 A schematic diagram of the projection of the target scattering points is shown;
[0114] Figure 3 A schematic diagram of the projection of two scattering points of the target is shown;
[0115] Figure 4A flow chart of a resource scheduling method for multi-target three-dimensional imaging of networked radars based on deep reinforcement learning is shown;
[0116] Figure 5 The radar and target spatial geographic location distribution map is shown;
[0117] Figure 6 A target motion direction diagram is shown;
[0118] Figure 7 A reference scatter point model diagram is shown;
[0119] FIG8(a) shows a scattering point model diagram of target 1, FIG8(b) shows a scattering point model diagram of target 2, FIG8(c) shows a scattering point model diagram of target 3, FIG8(d) shows a scattering point model diagram of target 4, FIG8(e) shows a scattering point model diagram of target 5, and FIG8(f) shows a scattering point model diagram of target 6;
[0120] Fig. 9 Shows a schematic diagram of the decision network and value network structure;
[0121] Figure 10(a) shows the value of the reward for each round of reinforcement learning training, and Figure 10(b) shows the state value of the final state of each round of reinforcement learning training;
[0122] Figure 11 shows three 2D ISAR images of target 1;
[0123] Figure 12(a) shows the three-dimensional reconstructed image of target 1, Figure 12(b) shows the XY plane projection image of the three-dimensional reconstructed structure of target 1, Figure 12(c) shows the XZ plane projection image of the three-dimensional reconstructed structure of target 1, and Figure 12(d) shows the YZ plane projection image of the three-dimensional reconstructed structure of target 1. DETAILED DESCRIPTION
[0124] The present invention will be further described below in conjunction with embodiments and drawings.
[0125] A resource scheduling method for multi-target 3D imaging of networked radars based on deep reinforcement learning considers the impact of the distribution of radar nodes observing the target on the quality of target 3D imaging, analyzes the demand for radar resources for target 3D imaging, allocates radar nodes to each target, and minimizes radar resource consumption while ensuring imaging quality. The method specifically includes the following steps:
[0126] Step 1: Perceive the target features and calculate the required azimuth resolution of the target;
[0127] Step 2: Determine the two-dimensional imaging plane and projection relationship of each radar to the target;
[0128] Step 3: Analyze the impact of radar node distribution on target 3D imaging quality;
[0129] Step 4: Construct a resource scheduling model for multi-target 3D imaging of networked radars;
[0130] Step 5: Design key elements of reinforcement learning;
[0131] Step 6: Solve the resource scheduling model for multi-target three-dimensional imaging of networked radars, and perform three-dimensional imaging of the target based on the scheduling results.
[0132] Further, the first step is specifically:
[0133] Through the radar tracking results of each target in the detection area, the geographical location, movement speed, distance and yaw angle of each target are obtained; then each radar emits a small number of pulses to perform two-dimensional ISAR coarse imaging of the target to obtain the imaging result s kr (f s ,f τ ), where f s is the fast time dimension frequency, f τ is the slow time dimension frequency, thus estimating the azimuth size S of the target kx and azimuthal sparsity K k ;
[0134] First, the ISAR image is normalized:
[0135]
[0136] Then set a threshold T s , T s is 0.5, According to the linear relationship between the Doppler frequency and the horizontal coordinate of the target scattering point Calculate the target azimuth size; where f d is the Doppler frequency generated by the relative motion between the target scattering point and the radar, v p is the radial velocity of the scattering point, λ is the wavelength of the transmitted signal, ω is the equivalent rotational angular velocity of the target, x p is the horizontal coordinate of the scattering point; the target azimuth size is expressed as:
[0137]
[0138] Then the ISAR image is normalized as follows:
[0139]
[0140] Set a threshold T h , Th is 0.2, and s″ kr (f τ ) is discretized into a vector s″ kr , then the azimuth sparsity K of the target k is the vector s″ kr Medium greater than T h The number of elements of ;
[0141] Assume the reference size is S x_ref , the reference azimuth resolution is ρ ref , then according to the target azimuth size, the required azimuth resolution of the target is:
[0142]
[0143] Further, the second step is specifically as follows:
[0144] First, the radar line of sight (LOS) vector is determined based on the geographic location of the radar and the target. Among them, P k , P r are the geographical locations of the target and radar respectively;
[0145] Then the target movement speed v k Decompose along the radar LOS direction and its orthogonal direction to obtain the target radial velocity v p With tangential velocity v t , where the tangential velocity is expressed as:
[0146]
[0147] For the three-dimensional space where the target is located, a three-dimensional rectangular coordinate system (O, X, Y, Z) is established with the target center O as the origin; for the imaging plane, such as Figure 1 As shown in the figure, the W axis is established along the radar LOS direction, and the U axis is established along the target tangential velocity direction. The unit vectors of the two axes are u u =v t / |v t |, then the radar imaging plane is represented as (O,U,W);
[0148] Next, the projection relationship is derived. Under far-field conditions, the target ISAR image is considered to be the projection of the target's three-dimensional structure on the imaging plane; Figure 2 As shown, point P is a point outside the imaging plane, and point Q is the projection point of point P on the imaging plane. The two unit vectors perpendicular to the plane are expressed as:
[0149]
[0150] Let the three-dimensional coordinates of point P be (x, y, z) T , the two-dimensional coordinates of point Q are (u,w) T ,but Expressed as Express equation (6) in the form of matrix multiplication:
[0151]
[0152] Then the projection relationship between the three-dimensional coordinates of the scattering point and the two-dimensional coordinates on the imaging plane is constructed:
[0153]
[0154] in, It is the projection matrix from the three-dimensional coordinates of a point in space to the imaging plane.
[0155] Further, the third step is specifically:
[0156] The present invention adopts a three-dimensional imaging method based on imaging projection to image the target. The main idea is to select multiple radars to perform two-dimensional ISAR imaging of the target at different sites and different viewing angles, and then reconstruct the three-dimensional scattering point structure of the target based on the obtained multiple two-dimensional ISAR images and their corresponding projection relationships; according to the idea, a size N is designed. x ·N y ·N z The three-dimensional Boolean matrix X is used as the optimization variable, where the matrix value 1 represents the existence of a target scattering point at that point. According to the peak value of the ISAR image intensity, the strong scattering points in the two-dimensional ISAR image of the target formed by the i-th radar are extracted and expressed as G i , the projection image of the target on the imaging plane is represented as F i (X), where the projection function F i (·) can be obtained based on the projection matrix φ of the imaging plane. The optimization function is constructed with the goal of minimizing the difference between the target 2D ISAR image and the projection image:
[0157]
[0158] Among them, N R Indicates that there are a total of N R The target is imaged by a radar, and λ is a regularization parameter. By solving the optimization model, the three-dimensional scattering point structure of the target can be reconstructed.
[0159] Then the influence of radar node distribution on target imaging quality is analyzed.
[0160] A single two-dimensional image cannot reveal the three-dimensional structure of the target. In order to locate the three-dimensional scattering points of the target, each target must be imaged by at least two radars. Suppose there are two points P1 and P2 in space, such as Figure 3 As shown, there are two radars imaging the target, and the projection points of P1 and P2 on the two imaging planes are Q i1 , Q i2 , Q j1 , Q j2 ,set up The normal vector of the imaging plane i is n i ,θ i for With n i The angle between i1 With Q i2 The distance between is expressed as:
[0161]
[0162] When l i ≥ρ i When P1 and P2 can be distinguished on the imaging plane i, ρ i is the two-dimensional resolution of imaging plane i, and its equivalent condition is: If two points can be distinguished on at least one imaging plane, correct reconstruction of the two points can be achieved.
[0163] Therefore, the condition for achieving correct reconstruction of the target is expressed as:
[0164]
[0165] C0 is the target scattering point set, and N is the number of radars observing the target. This condition is transformed into something related only to the radar node distribution, expressed as:
[0166]
[0167] where θ ij is the angle between the two radar imaging planes, ρ rec is the target 3D resolution, θ ij Obtained by the angle between the normal vectors of the two imaging planes:
[0168] The normal vector n of the i-th plane and the j-th plane is obtained by cross-multiplying the unit vector basis of the imaging plane. i 、n j , the angle between the normal vectors of the imaging planes is the angle between the imaging planes:
[0169]
[0170] If the two observation radars meet the above conditions, the target scattering point pairs can be distinguished on the imaging plane of at least one of the radars, so the correct reconstruction of the target can be achieved.
[0171] Furthermore, the influence of the number of observation radars on the imaging quality is analyzed.
[0172] Assume that the correct reconstruction result and a certain wrong reconstruction result are X and X', and bring the reconstruction result into the loss function in the optimization model of formula (9). In order to ensure that the final solution of the model is the correct reconstruction result of the target, the following conditions need to be met:
[0173]
[0174] Assume that X contains two scattering points P1 and P2, and X' contains only one of the scattering points. There are N planes that can correctly distinguish the two scattering points P1 and P2. P , then:
[0175]
[0176] The first half of the formula is negative, so it can be seen that N P The larger the value, the greater the probability that the above inequality holds. For most scattering point pairs, the more radars that image the target, the greater the number of imaging planes N that can correctly distinguish them. P Therefore, to this extent, we believe that the more radars observe a single target, the lower the probability of the target being reconstructed incorrectly, and the higher the target imaging quality.
[0177] Further, the fourth step is specifically:
[0178] Based on the above analysis, a resource scheduling model for multi-target three-dimensional imaging of networked radars is constructed to assign radars to each target for three-dimensional imaging. Under the premise of ensuring a certain imaging quality, the resource consumption of networked radars is minimized.
[0179] First of all, there are three goals to clarify the problem:
[0180] 1. Realize three-dimensional imaging of all targets;
[0181] 2. Ensure that each target can meet certain imaging quality requirements;
[0182] 3. The total time used by the networked radar for three-dimensional imaging of multiple targets is the shortest.
[0183] In order to avoid solving complex multi-objective optimization problems, in the resource scheduling model of the present invention, we set the shortest total scheduling time as our objective function, and the requirements for target imaging are reflected in the constraints. The constraints are set as the following three points:
[0184] 1. Each target has at least two radars that can meet the conditions for reconstructing any scattering point to image it, that is, there are two radars that meet the conditions of formula (12);
[0185] 2. The number of radars observing the target is as large as possible to reduce the probability of erroneous reconstruction. Each target is imaged by at least three radars;
[0186] 3. Each radar can use the sparse ISAR imaging algorithm based on compressed sensing to observe multiple targets at the same time, so the pulse sequence assigned to each target needs to meet the sparse reconstruction requirements.
[0187] Suppose there are M radars and N targets to be imaged. We define an M*N dimensional Boolean matrix W as the optimization variable to represent the scheduling scheme. W(m,n)=1 represents that the mth radar images the nth target. Suppose the total time used by the networked radar to complete the multi-target 3D imaging is T, which can be solved by the following steps:
[0188] Step 4.1 Extract from the scheme W which targets each radar needs to observe and record them in the radar's target set TSet(m);
[0189] Step 4.2 Prioritize the target based on the target flight speed, the distance between the target and the radar, and the yaw angle. k Perform a comprehensive evaluation, and then calculate the coherent integration time T required for imaging according to the azimuth resolution required for target imaging. k and observation dimension M k ;
[0190] Step 4.3 Each radar allocates sparse pulses to the target in turn according to the priority until imaging of all targets in the target set is completed;
[0191] Step 4.4 Calculate the time T that each radar consumes to complete the 2D ISAR imaging of all targets in its target set. m , and then select the longest time as the total time T used by the networked radar to complete multi-target three-dimensional imaging.
[0192] The above steps are defined as Tconsum(W), and the objective function is expressed as:
[0193] min Tconsum(W) (16)
[0194] Then determine whether solution W meets the target reconstruction conditions.
[0195] Assuming that the radar observes all targets, the angle between the normal vectors of the imaging planes of each two radars observing N targets can be calculated, and the minimum angle between the two radar imaging planes that meets the reconstruction condition can be calculated according to formula (12), and they are recorded in The matrix θ of dimension RT and θ pho middle, Represents a total of A combination of radar nodes.
[0196] Then, for each column of W, the elements are sorted according to θ. RT and θ ρ Perform bitwise AND operations between two elements in the order of dimensional matrix RT, RT(k,n) = 1 means that target n is observed by two radars in the kth radar pair. Then determine whether the scheme W meets the reconstruction conditions of each target:
[0197] Satis=[(RT⊙θ RT )>θ pho ] (17)
[0198] If any column in Satis has a radar combination that meets the conditions, that is, It is believed that scheme W can achieve the correct reconstruction of each target. At the same time, in order to reduce the probability of incorrect reconstruction, we hope that each target has at least 3 radars observing it.
[0199] To sum up, the resource scheduling model can be listed:
[0200] min Tconsum(W)
[0201] stn=1,2,...,N
[0202] sum(W(:,n))≥3
[0203] RT=W(i,n)&W(j,n),i,j=1,2,...,M
[0204] Satis=[(RT☉θ RT )>θ pho ]
[0205] sum(Satis(:,n))>=1 (18)
[0206] Further, the fifth step is specifically:
[0207] The present invention adopts the Advantage actor-critic (A2C) deep reinforcement learning method to solve the resource scheduling model. The key elements of reinforcement learning are designed as follows:
[0208] Step 5.1 State: Each radar node allocation scheme is considered as a State, and the radar node selection scheme for each target is S n , then State is the radar node selection scheme for all targets: S = (S1, S2, ..., S n ), where S n is a Boolean vector of length m, for example, S1 = (0, 1, 1, 0) means there are 4 radars in total, and target 1 is observed by radar 2 and radar 3;
[0209] Step 5.2 Action: Set the action to target number n + radar node selection scheme S k :a=(n,S k ), that is, each action only changes the radar node selection scheme of one target, among which some radar node selection schemes that do not meet the conditions will be screened out according to the constraints of the resource scheduling model, and the action space will be pruned;
[0210] Step 5.3 State Transition: Directly according to action a=(n,S k ) Change the radar node selection scheme of target n to S k ;
[0211] Step 5.4 State Value: Keep the state value consistent with the objective function of the scheduling model and set it to the negative value of the total time it takes for the networked radar to complete the multi-target three-dimensional imaging task;
[0212] Step 5.5 Reward: In reinforcement learning, rewards are used to convey the goals you want to achieve. Since the goal of reinforcement learning is to maximize the cumulative return, and the goal of the resource scheduling model we set is to find the state with the maximum state value, we set the reward for each action to the state value of the next state minus the state value of the current state, that is:
[0213] Reward=SV(S n )-SV(S c ) (19)
[0214] Among them, S n represents the next state, S crepresents the current state, and SV(·) represents the state value. When the discount factor γ is set to a number close to 1, the cumulative return of each round of training is approximately the state value of the last state of the training round minus the state value of the initial state, that is:
[0215]
[0216] Among them, r is the reward value. From the formula, it can be seen that the state value SV(S0) of the initial state is certain, and the state with the maximum state value can be obtained by using the reinforcement learning method.
[0217] Further, the sixth step is specifically:
[0218] The present invention uses the A2C deep reinforcement learning method to solve the resource scheduling model. Actor-Critic is a classic policy-based deep reinforcement learning method, which consists of two networks, Actor and Critic. Actor is a decision-making network that can make decisions and actions based on the input state, while Critic is a value network used to evaluate the quality of selected actions. The Actor-Critic method continuously interacts with the environment, obtains feedback, and then uses the feedback information to train the parameters of the two networks, and finally achieves network convergence. The A2C method introduces a baseline based on the Actor-Critic method, which further reduces the estimated variance of the action value.
[0219] Since the resource scheduling problem is an optimization problem, we need to first transform the problem into a Markov decision process. In the action design, we design each action to only change the radar node selection scheme of a target. Therefore, before finally obtaining the optimal solution, we will perform several actions from an initial state to form a decision process. Since the optimal solution is unknown, we dynamically set the end point of each training, and use the best solution currently obtained as the end point of this round of training. When we obtain the current optimal solution or find a solution with a higher state value, we will end this round of training. In summary, we transform the problem into a Markov decision process, and then use the A2C method to solve the model.
[0220] The specific steps of solving the resource scheduling model using the A2C method are as follows:
[0221] Step 6.1 Initialize the policy network parameters Θ, value network parameters Ω, best_value = -10000
[0222] Step 6.2 for the number of training rounds from 1 to 500:
[0223] Step 6.3 done=False
[0224] Step 6.4 while not done:
[0225] Step 6.5 Based on the strategy π(Θ) and the current state S c Get action a
[0226] Step 6.6 After taking action, get the next state S n and rewards n
[0227] Step 6.7 Calculate TD error and update the two network parameters Θ and Ω
[0228] Step 6.8 if SV(S n )>best_value
[0229] Step 6.9 best_value=SV(S n )
[0230] Step 6.10 end if
[0231] Step 6.11 if SV(S n )==best_value
[0232] Step 6.12 done=True
[0233] Step 6.13 end if
[0234] Step 6.14 S c =S n
[0235] Step 6.15 end while
[0236] Step 6.16 end for
[0237] After the algorithm converges, the optimal radar node scheduling plan can be obtained. Then, each radar allocates pulses to the targets in its target set according to the priority according to the scheduling plan for imaging. Finally, the three-dimensional image of each target is reconstructed according to the three-dimensional imaging algorithm.
[0238] Simulation Example
[0239] Figure 4A flow chart of the resource scheduling method for multi-target three-dimensional imaging of networked radars based on deep reinforcement learning is shown. The algorithm first perceives the target features, determines the two-dimensional imaging plane and projection relationship of each radar to the target, and then analyzes the impact of radar node distribution on the target three-dimensional imaging quality; on this basis, a resource scheduling model for multi-target three-dimensional imaging of networked radars is constructed, the key elements of reinforcement learning are designed, and the A2C deep reinforcement learning method is used to solve the resource scheduling model; finally, each radar performs two-dimensional ISAR imaging of the target according to the scheduling results, and a three-dimensional imaging algorithm based on imaging projection is used according to multiple two-dimensional ISAR images and their corresponding projection relationships to realize the reconstruction of the target's three-dimensional scattering points.
[0240] Assume that the radar transmits a linear frequency modulation signal with a carrier frequency of f c =10GHz, bandwidth is B=300MHz, distance resolution is ρ r =0.5m, pulse repetition frequency PRF=1000Hz, pulse width T p =1μs, target reference azimuth dimension S x_ref =25m, the required azimuth resolution is ρ ref =0.5m, the target three-dimensional resolution is ρ rec =1m.
[0241] A network radar consisting of 5 radars was set up to observe 6 targets within the observation range. Their spatial geographical location distribution and target movement direction are as follows Figure 5 , Figure 6 The spatial position coordinates of the radar and the target and the target velocity vector are shown in Table 2 and Table 3 respectively.
[0242] Table 2 Radar and target space position coordinates
[0243]
[0244] Table 3 Target velocity vector table
[0245]
[0246] The three-dimensional scattering point model of the target is unified as Figure 7 The scattering point model of is used as a reference, and the corresponding rotation is performed around the center of the scattering point model according to the target movement direction. The scattering point models of the six targets are shown in Figure 8.
[0247] The method for scheduling resource of multi-target three-dimensional imaging of networked radar based on deep reinforcement learning is specifically as follows:
[0248] Step 1: Perceive the target features and calculate the required azimuth resolution of the target
[0249] Through the radar tracking results of each target in the detection area, the geographical location, movement speed, distance and yaw angle of each target are obtained; then each radar emits a small number of pulses to perform two-dimensional ISAR coarse imaging of the target, and the azimuth size S of the target in the observation angle of each radar can be estimated by analyzing the image. kx and azimuthal sparsity K k , and the results are shown in Table 4 and Table 5 respectively.
[0250] Table 4 Target azimuth dimensions (m)
[0251] Goal 1 Goal 2 Goal 3 Goal 4 Goal 5 Goal 6 Radar 1 16.38 18.14 19.80 16.86 14.86 18.03 Radar 2 19.12 19.93 16.88 19.65 19.99 18.83 Radar 3 18.59 19.78 18.58 17.59 16.64 15.26 Radar 4 17.52 18.44 19.54 18.35 18.60 18.37 Radar 5 18.36 17.16 16.46 19.21 19.05 19.86
[0252] Table 5 Target sparsity
[0253] Goal 1 Goal 2 Goal 3 Goal 4 Goal 5 Goal 6 Radar 1 45 41 32 54 53 33 Radar 2 46 23 43 46 42 37 Radar 3 44 35 48 45 52 47 Radar 4 33 44 24 42 36 38 Radar 5 48 38 41 45 42 38
[0254] Step 2: Determine the two-dimensional imaging plane and projection relationship of each radar to the target
[0255] First, the radar line of sight (LOS) vector is determined based on the geographic location of the radar and the target. Then the target movement speed v k By decomposing along the radar LOS direction and its orthogonal direction, the target radial velocity v can be obtained. p With tangential velocity v t , where the tangential velocity is expressed as:
[0256]
[0257] The W axis is established along the radar LOS direction, and the U axis is established along the target tangential velocity direction. The unit vectors of the two axes are u u =v t / |v t |, then the radar imaging plane can be expressed as (O,U,W).
[0258] Under far-field conditions, the target ISAR image can be considered as the projection of the target's three-dimensional structure on the imaging plane. The projection relationship between the three-dimensional coordinates of the scattering point and the two-dimensional coordinates on the imaging plane can be determined as:
[0259]
[0260] in, It is the projection matrix from the three-dimensional coordinates of a point in space to the imaging plane.
[0261] Step 3: Analyze the impact of radar node distribution on target 3D imaging quality
[0262] The present invention uses a three-dimensional imaging method based on imaging projection to image the target. The main idea is to select multiple radars to perform two-dimensional ISAR imaging of the target at different sites and different viewing angles, and then reconstruct the three-dimensional scattering point structure of the target based on the multiple two-dimensional ISAR images obtained and their corresponding projection relationships. According to this idea, a size N is designed. x ·N y ·N z The three-dimensional Boolean matrix X is used as the optimization variable, where the matrix value 1 represents the existence of a target scattering point at that point. According to the peak value of the ISAR image intensity, the strong scattering points in the two-dimensional ISAR image of the target formed by the i-th radar are extracted and expressed as G i , the projection image of the target on the imaging plane is represented as F i (X), where the projection function F i (·) can be obtained based on the projection matrix φ of the imaging plane. The optimization function is constructed with the goal of minimizing the difference between the target 2D ISAR image and the projection image:
[0263]
[0264] Among them, N R Indicates that there are a total of N R The target is imaged by a radar, and λ is a regularization parameter. By solving the optimization model, the three-dimensional scattering point structure of the target can be reconstructed.
[0265] Then the influence of radar node distribution on the target three-dimensional imaging quality is analyzed.
[0266] A single two-dimensional image cannot reveal the three-dimensional structure of the target. In order to locate the three-dimensional scattering points of the target, each target must be imaged by at least two radars. Suppose there are two points P1 and P2 in space, such as Figure 3 As shown, there are two radars imaging the target, and the projection points of P1 and P2 on the two imaging planes are Q i1 , Q i2 , Q j1 , Q j2 ,set up The normal vector of the imaging plane i is n i ,θ i for With n i The angle between i1 With Q i2 The distance between can be expressed as:
[0267]
[0268] When l i ≥ρ i When P1 and P2 can be distinguished on the imaging plane i, ρi is the two-dimensional resolution of imaging plane i, and its equivalent condition is: If two points can be distinguished on at least one imaging plane, correct reconstruction of the two points can be achieved.
[0269] Therefore, the condition for achieving correct reconstruction of the target can be expressed as:
[0270]
[0271] C0 is the target scattering point set, and N is the number of radars observing the target. This condition is transformed into something related only to the distribution of radar nodes, which can be expressed as:
[0272]
[0273] where θ ij is the angle between the two radar imaging planes, ρ rec is the target 3D resolution. ij It can be obtained by the angle between the normal vectors of the two imaging planes:
[0274] The normal vector n of the i-th plane and the j-th plane is obtained by cross-multiplying the unit vector basis of the imaging plane. i 、n j , the angle between the normal vectors of the imaging planes is the angle between the imaging planes:
[0275]
[0276] If the two observation radars meet the above conditions, the target scattering point pairs can be distinguished on the imaging plane of at least one of the radars, so the correct reconstruction of the target can be achieved.
[0277] Furthermore, we analyze the impact of the number of observing radars on the imaging quality.
[0278] Assume that the correct reconstruction result and a certain wrong reconstruction result are X and X', and bring the reconstruction result into the loss function in the optimization model of formula (3). In order to ensure that the final solution of the model is the correct reconstruction result of the target, the following conditions need to be met:
[0279]
[0280] Assume that X contains two scattering points P1 and P2, and X' contains only one of the scattering points. There are N planes that can correctly distinguish the two scattering points P1 and P2. P , then:
[0281]
[0282] The first half of the formula is negative, so it can be seen that NP The larger the value, the greater the probability that the above inequality holds. For most scattering point pairs, the more radars that image the target, the greater the number of imaging planes N that can correctly distinguish them. P Therefore, to this extent, we believe that the more radars observe a single target, the lower the probability of the target being reconstructed incorrectly, and the higher the target imaging quality.
[0283] Step 4: Constructing a resource scheduling model for multi-target 3D imaging of networked radars
[0284] A resource scheduling model for multi-target three-dimensional imaging of networked radars is constructed to assign radars to each target for three-dimensional imaging, thereby minimizing the resource consumption of networked radars while ensuring a certain imaging quality.
[0285] In the embodiment, there are 5 radars and 6 targets to be imaged. A 5*6-dimensional Boolean matrix W is defined as an optimization variable to represent the scheduling scheme. W(m,n)=1 represents that the mth radar images the nth target. Assume that the total time used by the networked radar to complete multi-target 3D imaging is T, which can be solved by the following steps:
[0286] 1. Extract from the scheme W which targets each radar needs to observe and record them in the radar's target set TSet(m);
[0287] 2. Target priority p is determined based on the target flight speed, the distance between the target and the radar, and the yaw angle k Perform a comprehensive evaluation, and then calculate the coherent integration time T required for imaging according to the azimuth resolution required for target imaging. k and observation dimension M k ;
[0288] 3. Each radar allocates sparse pulses to the target in turn according to the priority until all targets in the target set are imaged;
[0289] 4. Calculate the time T that each radar consumes to complete the 2D ISAR imaging of all targets in its target set. m , and then select the longest time as the total time T used by the networked radar to complete multi-target three-dimensional imaging.
[0290] The above steps are defined as Tconsum(W), and the objective function can be expressed as:
[0291] min Tconsum(W) (10)
[0292] Assuming that the radar can observe all targets, the angle between the normal vectors of the imaging planes of each two radars observing the six targets can be calculated, and the minimum angle between the two radar imaging planes that meets the reconstruction condition can be calculated according to formula (12), and they are recorded in The matrix θ of dimension RT and θ ρ middle, Represents a total of A combination of radar nodes.
[0293] Then, for each column of W, the elements are sorted according to θ. RT and θ pho Perform bitwise AND operations between two elements in the order of dimensional matrix RT, RT(k,n) = 1 means that target n is observed by two radars in the kth radar pair. Then determine whether the scheme W meets the reconstruction conditions of each target:
[0294] Satis=[(RT⊙θ RT )>θ pho ] (11)
[0295] If any column in Satis has a radar combination that meets the conditions, that is, It is believed that scheme W can achieve the correct reconstruction of each target. At the same time, in order to reduce the probability of incorrect reconstruction, we hope that each target has at least 3 radars observing it.
[0296] To sum up, the resource scheduling model can be listed:
[0297] min Tconsum(W)
[0298] stn=1,2,...,N
[0299] sum(W(:,n))≥3
[0300] RT=W(i,n)&W(j,n),i,j=1,2,...,M
[0301] Satis=(RT☉θ RT )>θ pho ]
[0302] sum(Satis(:,n))>=1 (12)
[0303] Step 5: Design key elements of reinforcement learning
[0304] The present invention adopts Advantage actor-critic (A2C) deep reinforcement learning method to solve the resource scheduling model. The key elements of reinforcement learning are designed as follows:
[0305] 1. State: Each radar node allocation scheme is considered as a State. Suppose the radar node selection scheme for each target is S n , then State is the radar node selection scheme for all targets: S = (S1, S2, ..., S n ), where S n is a Boolean vector of length m, for example, S1 = (0, 1, 1, 0) means there are 4 radars in total, and target 1 is observed by radar 2 and radar 3;
[0306] 2. Action: Set the action to target number n + radar node selection scheme S k :a=(n,S k ), that is, each action only changes the radar node selection scheme of one target. Among them, some radar node selection schemes that do not meet the conditions will be screened out according to the constraints of the resource scheduling model, and the action space will be pruned;
[0307] 3. State Transition: Directly according to action a=(n,S k ) Change the radar node selection scheme of target n to S k ;
[0308] 4. State Value: Keep the state value consistent with the objective function of the scheduling model and set it to the negative value of the total time it takes for the networked radar to complete the multi-target 3D imaging task;
[0309] 5. Reward: Set the reward for each action to the state value of the next state minus the state value of the current state, that is:
[0310] Reward=SV(S n )-SV(S c ) (13)
[0311] Among them, S n represents the next state, S c represents the current state, and SV(·) represents the state value. When the discount factor γ is set to a number close to 1, the cumulative return of each round of training can be approximated by the state value of the last state of the training round minus the state value of the initial state, that is:
[0312]
[0313] Among them, r is the reward value. From the formula, it can be seen that the state value SV(S0) of the initial state is certain, and the state with the maximum state value can be obtained by using the reinforcement learning method.
[0314] Step 6: Solve the resource scheduling model for multi-target 3D imaging of networked radars, and perform 3D imaging of the targets based on the scheduling results.
[0315] The present invention uses the A2C deep reinforcement learning method to solve the resource scheduling model. The decision network and the value network use a three-layer fully connected neural network, which is the input layer, the hidden layer and the output layer. The number of neurons in the hidden layer is 128, and ReLU is used as the activation function. The network structure is as follows: Fig. 9 shown.
[0316] The decision network learning rate is set to lr_a=1*10 -3 , the value network learning rate is set to lr_v=1*10 -3 , the value discount rate is set to γ = 0.98, and the total number of iterations is set to 500.
[0317] The network training results are shown in Figure 10(a) and Figure 10(b). From the figures, we can see that the algorithm found the optimal solution after about 100 rounds of training, and then the network basically converged.
[0318] The best scheduling solution is shown in Table 6.
[0319] Table 6 Best scheduling solution
[0320]
[0321] The total imaging time obtained by this scheme is 2.055 seconds, and the pulse utilization rates of each radar are 0.75, 0.88, 0.73, 0.52, and 0.63, respectively.
[0322] Each radar images the target according to the scheme, obtains multiple 2D ISAR images of the target from different viewing angles, and then reconstructs the target 3D structure using multiple 2D ISAR images and their corresponding projection relationships according to the target 3D imaging algorithm. The mean square error of the reconstruction results of each target is shown in Table 7.
[0323] Table 7 Mean square error (MSE) of each target reconstruction
[0324]
[0325] Among them, three two-dimensional imaging results of target 1 are shown in Figure 11, and the three-dimensional imaging result of target 1 is shown in Figure 12. It can be seen from the target image and the MSE index that the quality of the imaging result finally obtained by the algorithm proposed in the present invention can meet the requirements.
[0326] The present invention further compares the performance of the proposed algorithm with an existing resource scheduling algorithm for multi-target three-dimensional imaging of networked radars. Algorithm 2 constructs a resource scheduling model with the shortest total imaging time as the goal, requiring each target to be observed by three non-collinear radars. In addition, the impact of radar node distribution on imaging quality is not considered too much.
[0327] The scheduling solution obtained by using Algorithm 2 to solve the scenario of this embodiment is shown in Table 8.
[0328] Table 8 Algorithm 2 scheduling scheme
[0329]
[0330] The total imaging time obtained by this scheme is 2.055 seconds, and the pulse utilization rates of each radar are 0.74, 0.67, 0.55, 0.52, and 0.41, respectively, which are generally lower than the radar pulse utilization rate of the algorithm proposed in the present invention.
[0331] The mean square error (MSE) of the target reconstruction obtained by the two algorithms is shown in Table 9.
[0332] Table 9 Mean square error (MSE) of target reconstruction by two algorithms
[0333]
[0334] It can be seen from the data in the table that for targets 4 and 5, the imaging effect of the targets obtained by the scheduling results obtained by the proposed algorithm is significantly better than that of Algorithm 2, thus proving the effectiveness of the algorithm proposed in the present invention.
[0335] The advantages of the present invention are as follows: 1. The influence of radar node distribution on the target three-dimensional imaging quality is considered, and a network radar multi-target three-dimensional imaging resource scheduling model is constructed to minimize time resource consumption while ensuring imaging quality. 2. The model is solved by using a deep reinforcement learning method to obtain the optimal solution to the problem.
Claims
1. A method for resource scheduling of multi-target three-dimensional imaging of networked radar based on deep reinforcement learning, specifically comprising the following steps: Step 1: Perceive the target features and calculate the required azimuth resolution of the target; Step 2: Determine the two-dimensional imaging plane and projection relationship of each radar to the target; Step 3: Analyze the impact of radar node distribution on target 3D imaging quality; Step 4: Construct a resource scheduling model for multi-target 3D imaging of networked radars; Step 5: Design key elements of reinforcement learning; Step 6: Solve the resource scheduling model for multi-target three-dimensional imaging of networked radars, and perform three-dimensional imaging of the target based on the scheduling results.
2. The method for scheduling networked radar multi-target three-dimensional imaging resources based on deep reinforcement learning as claimed in claim 1, wherein the first step is specifically: Through the radar tracking results of each target in the detection area, the geographical location, movement speed, distance and yaw angle of each target are obtained; then each radar emits a small number of pulses to perform two-dimensional ISAR coarse imaging of the target to obtain the imaging result s kr (f s ,f τ ), where f s is the fast time dimension frequency, f τ is the slow time dimension frequency, thus estimating the azimuth size S of the target kx and azimuthal sparsity K k ; First, the 2D ISAR coarse imaging is normalized: Then, set the threshold T s ,remember According to the linear relationship between the Doppler frequency and the horizontal coordinate of the target scattering point Calculate the target azimuth size; where f d is the Doppler frequency generated by the relative motion between the target scattering point and the radar, v p is the radial velocity of the scattering point, λ is the wavelength of the transmitted signal, ω is the equivalent rotational angular velocity of the target, x p is the horizontal coordinate of the scattering point; the target azimuth size is expressed as: Then, the ISAR image is normalized as follows: Set the threshold T h , s″ kr (f τ ) is discretized into a vector s″ kr , then the azimuth sparsity K of the target k is the vector s″ kr Medium greater than T h The number of elements of ; Assume the reference size is S x_ref , the reference azimuth resolution is ρ ref , then according to the target azimuth size, the required azimuth resolution of the target is:
3. The method for scheduling multi-target three-dimensional imaging resources of a networked radar based on deep reinforcement learning as claimed in claim 1, wherein the threshold T in the first step is s is 0.5, the threshold T h is: 0.
2.
4. According to the method for scheduling resource of multi-target three-dimensional imaging of networked radar based on deep reinforcement learning as claimed in claim 3, the second step is specifically: Determine the radar line of sight direction vector based on the geographical location of the radar and the target in, P k , P r are the geographical locations of the target and radar respectively; The target movement speed v k Decompose along the radar line of sight and its orthogonal direction to obtain the target radial velocity v p With tangential velocity v t , where the tangential velocity is expressed as: For the three-dimensional space where the target is located, a three-dimensional rectangular coordinate system (O, X, Y, Z) is established with the target center O as the origin; for the imaging plane, the W axis is established along the radar line of sight, and the U axis is established along the target tangential velocity direction. The unit vectors of the two axes are u u =v t / |v t |, then the radar imaging plane is represented as (O,U,W); Next, the projection relationship is derived. Under far-field conditions, the target ISAR image is considered to be the projection of the target's three-dimensional structure on the imaging plane; point P is a point outside the imaging plane, and point Q is the projection point of point P on the imaging plane. Then The two unit vectors perpendicular to the plane are expressed as: Let the three-dimensional coordinates of point P be (x, y, z) T , the two-dimensional coordinates of point Q are (u,w) T ,but Expressed as Express equation (6) in the form of matrix multiplication: Then the projection relationship between the three-dimensional coordinates of the scattering point and the two-dimensional coordinates on the imaging plane is constructed: in, It is the projection matrix from the three-dimensional coordinates of a point in space to the imaging plane.
5. The method for scheduling networked radar multi-target three-dimensional imaging resources based on deep reinforcement learning as claimed in claim 1, wherein the third step is specifically: Design a size N x ·N y ·N z The three-dimensional Boolean matrix X is used as the optimization variable, where The matrix value of 1 represents the existence of a target scattering point at that point. The strong scattering point in the two-dimensional ISAR image of the target formed by the i-th radar is extracted according to the peak value of the ISAR image intensity and expressed as G i , the projection image of the target on the imaging plane is represented as F i (X), where the projection function F i (·) is obtained according to the projection matrix φ of the imaging plane; the optimization function is constructed with the goal of minimizing the difference between the target 2D ISAR image and the projection image: Among them, N R Indicates that there are a total of N R The target is imaged by a radar, and λ is a regularization parameter. The three-dimensional scattering point structure of the target can be reconstructed by solving the optimization model. Then the influence of radar node distribution on target imaging quality is analyzed; A single two-dimensional image cannot reveal the three-dimensional structure of the target. To locate the three-dimensional scattering points of the target, each target must be imaged by at least two radars. Suppose there are two points P1 and P2 in space. Two radars image the target. The projection points of P1 and P2 on the two imaging planes are Q i1 , Q i2 , Q j1 , Q j2 ,set up The normal vector of the imaging plane i is n i ,θ i for With n i The angle between i1 With Q i2 The distance between is expressed as: When l i ≥ρ i When P1 and P2 can be distinguished on the imaging plane i, ρ i is the two-dimensional resolution of imaging plane i, and its equivalent condition is: If two points can be distinguished on at least one imaging plane, correct reconstruction of the two points can be achieved; Therefore, the condition for achieving correct reconstruction of the target is expressed as: C0 is the target scattering point set, N is the number of radars observing the target; this condition is transformed into only the distribution of radar nodes, expressed as: where θ ij is the angle between the two radar imaging planes, ρ rec is the target 3D resolution, θ ij Obtained by the angle between the normal vectors of the two imaging planes; The normal vector n of the i-th plane and the j-th plane is obtained by cross-multiplying the unit vector basis of the imaging plane. i 、n j , the angle between the normal vectors of the imaging planes is the angle between the imaging planes: If the two observation radars meet the above conditions, the target scattering point pairs can be distinguished on the imaging plane of at least one of the radars, so the correct reconstruction of the target can be achieved. The influence of the number of observation radars on the imaging quality is further analyzed.
6. According to the method for scheduling multi-target three-dimensional imaging resources of networked radars based on deep reinforcement learning as claimed in claim 5, the further analysis of the influence of the number of observation radars on the imaging quality in the third step is specifically as follows: Assume that the correct reconstruction result and a certain wrong reconstruction result are X and X', and bring the reconstruction result into the loss function in the optimization model of formula (9). In order to ensure that the final solution of the model is the correct reconstruction result of the target, the following conditions need to be met: Assume that X contains two scattering points P1 and P2, and X' contains only one of the scattering points. There are N planes that can correctly distinguish the two scattering points P1 and P2. P , then: The first half of the formula is negative, so N P The larger the value, the greater the probability that the above inequality holds true; for most scattering point pairs, the more radars that image the target, the greater the number of imaging planes N that can correctly distinguish them. P Therefore, to this extent, we believe that the more radars observe a single target, the lower the probability of the target being reconstructed incorrectly and the higher the target imaging quality.
7. The method for scheduling networked radar multi-target three-dimensional imaging resources based on deep reinforcement learning as claimed in claim 1, wherein the fourth step is specifically: Construct a resource scheduling model for multi-target 3D imaging of networked radars, assign radars to each target to perform 3D imaging, and minimize the resource consumption of networked radars while ensuring a certain imaging quality. Taking the shortest total scheduling time as our objective function, we set the constraint conditions required for target imaging. Suppose there are M radars and N targets to be imaged. We define an M*N dimensional Boolean matrix W as the optimization variable to represent the scheduling scheme. W(m,n)=1 represents that the mth radar images the nth target. Suppose the total time used by the networked radar to complete the multi-target three-dimensional imaging is T, which is solved by the following steps: Step 4.1 Extract from the scheme W which targets each radar needs to observe and record them in the radar's target set TSet(m); Step 4.2 Prioritize the target based on the target flight speed, the distance between the target and the radar, and the yaw angle. k Perform a comprehensive evaluation, and then calculate the coherent integration time T required for imaging according to the azimuth resolution required for target imaging. k and observation dimension M k ; Step 4.3 Each radar allocates sparse pulses to the target in turn according to the priority until imaging of all targets in the target set is completed; Step 4.4 Calculate the time T that each radar consumes to complete the 2D ISAR imaging of all targets in its target set. m , and then select the longest time as the total time T used by the networked radar to complete multi-target three-dimensional imaging; The above steps are defined as Tconsum(W), and the objective function is expressed as: min Tconsum(W) (16) Then determine whether solution W meets the target reconstruction condition; Assuming that the radar observes all targets, the angle between the normal vectors of the imaging planes of each two radars observing N targets is calculated, and the minimum angle between the imaging planes of the two radars that meets the reconstruction condition is calculated according to formula (12), and they are recorded in The matrix θ of dimension RT and θ pho middle, Represents a total of Radar node combination method; Then, for each column of W, the elements are sorted according to θ. RT and θ ρ Perform bitwise AND operations between two elements in the order of dimensional matrix RT, RT(k,n) = 1 means that target n is observed by two radars in the k-th radar pair; Then determine whether solution W meets the reconstruction conditions of each goal: Enough=[(RT⊙θ RT )>θ pho ] (17) If any column in Satis has a radar combination that meets the conditions, that is, It is believed that scheme W can achieve the correct reconstruction of each target; at the same time, in order to reduce the probability of incorrect reconstruction, each target is observed by at least three radars; List resource scheduling models:
8. According to the resource scheduling method for multi-target three-dimensional imaging of networked radars based on deep reinforcement learning as described in claim 7, the fourth step sets the constraint conditions for target imaging requirements as follows:
1. Each target has at least two radars that can meet the conditions for reconstructing any scattering point to image it, that is, there are two radars that meet the condition of formula (12); 2. The number of radars observing the target is as large as possible to reduce the probability of erroneous reconstruction, and each target is imaged by at least three radars; 3. Each radar can use a sparse ISAR imaging algorithm based on compressed sensing to observe multiple targets at the same time, so the pulse sequence allocated to each target needs to meet the sparse reconstruction requirements.
9. The method for scheduling networked radar multi-target three-dimensional imaging resources based on deep reinforcement learning as claimed in claim 1, wherein the fifth step is specifically: The A2C method and deep reinforcement learning method are used to solve the resource scheduling model and design the key elements of reinforcement learning: Step 5.1 State: Each radar node allocation scheme is considered as a State, and the radar node selection scheme for each target is S n , then State is the radar node selection scheme for all targets: S = (S1, S2, ..., S n ), where S n is a Boolean vector of length m; Step 5.2 Action: Set the action to target number n + radar node selection scheme S k :a=(n,S k ), that is, each action only changes the radar node selection scheme of one target, where According to the constraints of the resource scheduling model, some radar node selection schemes that do not meet the conditions will be screened out and the action space will be pruned; Step 5.3 State Transition: Directly according to action a=(n,S k ) Change the radar node selection scheme of target n to S k ; Step 5.4 State Value: Keep the state value consistent with the objective function of the scheduling model and set it to the negative value of the total time it takes for the networked radar to complete the multi-target three-dimensional imaging task; Step 5.5 Reward: Set the reward for each action to the state value of the next state minus the state value of the current state, that is: Reward=SV(S n )-SV(S c ) (19) Among them, S n Indicates the next state, S c represents the current state, SV(·) represents the state value; when the discount factor γ is set to a number close to 1, the cumulative return of each round of training is approximately the state value of the last state of the training round minus the state value of the initial state, that is: Among them, r is the reward value. From formula (20), it can be seen that the state value SV(S0) of the initial state is certain, and the reinforcement learning method is used to solve the state with the maximum state value.
10. The method for scheduling networked radar multi-target three-dimensional imaging resources based on deep reinforcement learning as claimed in claim 1, wherein the sixth step is specifically: The specific steps of solving the resource scheduling model using the A2C method are as follows: Step 6.1 Initialize the policy network parameters Θ, value network parameters Ω, best_value = -10000 Step 6.2 for the number of training rounds from 1 to 500: Step 6.3 done=False Step 6.4 while not done: Step 6.5 According to the strategy π(Θ) and the current state S c Get action a Step 6.6 After taking action, get the next state S n and rewards n Step 6.7 Calculate the TD error and update the two network parameters Θ and Ω Step 6.8 if SV(S n )>best_value Step 6.9 best_value=SV(S n ) Step 6.10 end if Step 6.11 if SV(S n )==best_value Step 6.12 done=True Step 6.13 end if Step 6.14 S c =S n Step 6.15 end while Step 6.16 end for After the algorithm converges, the optimal radar node scheduling plan can be obtained. Then, each radar allocates pulses to the targets in its target set according to the priority according to the scheduling plan for imaging. Finally, the three-dimensional image of each target is reconstructed according to the three-dimensional imaging algorithm.
Citation Information
Patent Citations
Adaptive scheduling method for inverse synthetic aperture radar imaging resources in networking
CN108761455A
Resource self-adaptive scheduling method for inverse synthetic aperture radar two-dimensional sparse imaging
CN109459752A
Distributed networking radar node position optimization method based on reinforcement learning
CN116362122A
Multi-machine cooperative radar search resource optimization method based on multi-agent reinforcement learning
CN117709678A
ISAR imaging resource adaptive scheduling method based on optimal observation time period
CN117930239A
