Satellite optical image weak moving target simulation method

Abstract

The invention discloses a satellite optical image weak moving target simulation method, which can reproduce simulation targets with highly fit weak target forms such as civil aircrafts and the like on a medium-resolution optical image. Simple means such as two-dimensional image plane shape approximate simplified modeling and target abundance Gaussian blur processing are used for replacing tedious steps such as target three-dimensional appearance modeling and atmospheric transmission process modeling in existing simulation software, and the simulation difficulty and cost are greatly reduced. According to the method, a large-scale satellite image weak motion target data set can be automatically constructed and can be used as a training set of a target detection algorithm, and it is verified that a target detection network model trained by a simulation data set constructed based on the method can detect an effective and real civil aircraft target in a satellite image.

Description

Technical Field

[0001] The invention relates to the field of data simulation, in particular to a satellite optical image weak moving target simulation method. Background Art

[0002] As the aviation industry continues to prosper, the importance of real-time monitoring of global civil aircraft activities has become increasingly prominent. At present, civil aircraft traffic control mainly relies on ground-based radar and ADS-B system. Ground-based radar used for air traffic control usually has an operating distance of only a few hundred kilometers; the ADS-B system can passively receive the location information sent by civil aircraft through ground receiving equipment, and cannot cover most ocean areas. Satellite remote sensing images can provide global geographic images, and the number and location information of specified ground object types can be obtained through image detection algorithms. Medium spatial resolution satellite images (usually considered to have a ground sampling interval of 5-100m) can achieve an imaging width of hundreds of kilometers, and a single scene covers a wide range of the surface, which is suitable as a supplementary means for global civil aircraft activity monitoring tasks. However, the size of civil aircraft is close to the spatial resolution of medium spatial resolution satellite images, and it is difficult for human eyes to identify civil aircraft on some medium spatial resolution satellite images. Current data-driven target detection methods such as Faster RCNN and YOLO require a large number of sample labeled training sets, but it is difficult for the human eye to identify civil aircraft targets on medium spatial resolution satellite images, and it is difficult to manually label targets. Simulation technology can automatically generate simulated images with accurate target labels, which is helpful for the development of satellite image detection algorithms for weak moving targets such as civil aircraft targets.

[0003] The Vega series simulation module launched by Multigen-Paradigm of Germany is a widely used commercial remote sensing imaging simulation toolkit with relatively complete functions. It can realize dynamic and visual simulation of the entire remote sensing imaging process. The remote sensing imaging simulation software DIRSIG developed by the Digital Imaging and Remote Sensing (DIRS) Laboratory of Rochester Institute of Technology has undergone a series of upgrades and improvements. It can perform broadband, multi-spectral, and hyper-spectral imaging simulations from visible light to thermal infrared bands, and has been applied to many NASA projects. Simulation research on remote sensing images at home and abroad is generally a full-link simulation of the entire remote sensing imaging process. It is necessary to establish multiple models such as ground object scenes, target characteristics, atmospheric transmission, and detector response. The workload required is huge and the theory is complex. In addition, some remote sensing image simulation software has begun to be commercially applied, but the procurement cost is relatively high.

[0004] Satellite imagery weak moving target detection technology helps improve the ability of satellite optical images to monitor global civil aircraft activities and improve current civil aircraft control methods. The difficulty of manually annotating weak targets and the high price of existing remote sensing image simulation software make it difficult to build a satellite imagery weak moving target dataset, which in turn hinders the development of satellite imagery weak moving target detection technology.

[0005] The terms used in the present invention are explained as follows:

[0006] Abundance: reflects the coverage of a substance on the surface. The abundance of a substance in a pixel is the proportion of the substance in the pixel.

[0007] Linear mixture model: usually refers to the mixture of two or more signals or substances following the principle of linear superposition, that is, the total output or result of the mixture is a linear combination of the individual components. In a linear mixture, the contribution of each component is independent and can be calculated separately and then simply added together. In the field of remote sensing, linear mixture models are often used to explain the signals received by remote sensing sensors. These signals are the superposition of electromagnetic radiation reflected or emitted by different types of surface cover (such as vegetation, soil, water bodies, etc.). The linear mixture model assumes that the signal recorded by the sensor is a weighted average of the signals of various types of surface cover, and the weight is the proportion of each type of cover in the sensor's field of view. Summary of the invention

[0008] The technical problem to be solved by the present invention is to provide a method for simulating weak moving targets in satellite optical images in view of the shortcomings of the existing technology, so as to reproduce a simulated target with a highly consistent morphology of weak targets such as civil aircraft on a medium-resolution optical image.

[0009] In order to solve the above technical problems, the technical solution adopted by the present invention is: a satellite optical image weak moving target simulation method, comprising the following steps:

[0010] S1. Input a single satellite optical image Among them, H represents the image height, W represents the image width, and C represents the number of image channels;

[0011] S2, setting the simulation target speed, motion trajectory, and target starting point, and obtaining the coordinate value and heading angle of the target center point on the two-dimensional image plane;

[0012] S3, approximate and simplify the two-dimensional image plane shape of the modeling target to obtain an approximate shape model of the target;

[0013] S4. According to the target center point coordinate value and target heading angle, the target approximate shape model is embedded into the image and the target abundance matrix is ​​calculated.

[0014] S5, performing Gaussian blur processing on the target abundance matrix to simulate the point diffusion phenomenon in the imaging process, and obtaining the target abundance matrix after Gaussian blur;

[0015] S6. Inject the target full pixel radiation into the satellite optical image according to the target abundance matrix after Gaussian blur to generate a simulated target.

[0016] Target abundance A at pixel (i, j) i,j The calculation formula is:

[0017]

[0018] Among them, T i,j is the area occupied by the target approximate shape model on pixel (i, j), and P is the area of ​​a single pixel.

[0019] The pixel gray value V′ of the simulation target i ′ j The expression is as follows:

[0020] V′ i,j =(1-A i,j )V i,j +A i,j (1-r)V I ,;

[0021] Among them, V i,j is the radiance or radiance curve at pixel (i, j) on the original satellite optical image, A i,j is the target abundance after Gaussian blur at pixel (i, j), V I is the average radiation brightness value or average radiation brightness curve of the top M% pixels in the original infrared image, rV I is the reference radiation brightness value or radiation brightness curve of all target pixels, and r is the set scale factor.

[0022] In the present invention, r takes the value of [0,1].

[0023] In the present invention, M=30.

[0024] Compared with the prior art, the beneficial effects of the present invention are as follows: the present invention proposes a simple and easy method for simulating weak moving targets in satellite optical images, which can replicate simulated targets that are highly consistent with the measured morphology of weak targets such as civil aircraft on open source medium spatial resolution satellite images, and can automatically generate a large number of simulated samples and labels of weak moving targets in satellite images, which helps to meet the data requirements for the development of weak moving target detection algorithms in satellite images. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1This is a flow chart of a method according to an embodiment of the present invention;

[0026] Figure 2 A diagram of the steps involved in generating a target abundance matrix according to an embodiment of the present invention; (a) target shape modeling, (b) embedding the shape model into an image, (c) target abundance matrix calculation, and (d) Gaussian blurring of the abundance matrix;

[0027] Figure 3 To disclose the measured target 1 and its simulated target; (a) simulated image, (b) real target, (c) simulated target;

[0028] Figure 4 To disclose the measured target 2 and its simulated target; (a) simulated image, (b) real target, (c) simulated target;

[0029] Figure 5 The following are the results of the actual detection of civil aircraft targets in the air; (a) publicly measured target 1, (b) publicly measured target 2, (c) publicly measured target 3, and (d) publicly measured target 4. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0031] The embodiment of the present invention proposes a satellite optical image weak moving target simulation method, and the specific process is as follows:

[0032] S1. Single satellite optical image input;

[0033] S2. Set the motion characteristic information such as the simulation target speed, motion trajectory, target starting point, etc., and obtain the coordinate value and heading angle of the target center point on the two-dimensional image plane (the simulation single-frame image can be directly set);

[0034] S3. Simplified modeling of the target two-dimensional image plane shape approximation;

[0035] In satellite optical images, the shape of faint targets is blurred; in addition, satellite imaging of the surface usually adopts the nadir observation mode, that is, the photoelectric sensor directly images the surface area under the satellite. Therefore, this method approximates and simplifies the shape of the target's two-dimensional top view and constructs a two-dimensional image plane shape model. The fuselage length of current civil aircraft is close to the wingspan, so the sub-satellite observation shape of the civil aircraft is simplified to a cross with the same length and width, both of which are l, and the length of the outermost side of the cross is set to 0.3 of its length.

[0036] S4. embedding the target approximate shape model into the image and calculating the target abundance matrix;

[0037] According to the coordinates of the target center point and the heading angle, the simplified target shape model is pasted on the image. The area ratio of the target shape model on each pixel of the image is calculated, and this area ratio is used to construct a target abundance matrix A of the same size as the image. The target abundance A at pixel (i, j) i,j The calculation formula is:

[0038]

[0039] Among them, T i,j is the area occupied by the target approximate shape model on pixel (i, j), and P is the area of ​​a single pixel.

[0040] S5. Gaussian blur processing is performed on the target abundance matrix to simulate the nonlinear mixing phenomenon caused by atmospheric transmission;

[0041] Due to the scattering of light in the atmospheric propagation process, there is nonlinear mixing in the pixels of remote sensing images. This method uses the Gaussian blur algorithm (https: / / docs.opencv.org / 4.0.0 / d4 / d86 / group__imgproc__filter.html#gaabe8c836e97159a9193fb0b11ac52cf1) to perform image blur processing on the target abundance matrix, which approximately replaces the simulation steps of nonlinear effects such as atmospheric scattering. The standard deviation parameter of the Gaussian blur kernel in the Gaussian blur algorithm is set to σ.

[0042] S6. Setting the target full pixel reference radiance;

[0043] The target full pixel radiation is set according to the spectral radiation characteristics of the target in the band where the satellite image is located. According to the literature (Li L, Zhou X, Hu Z, et al. On-orbit monitoring flying aircraft day and night based on SDGSAT-1 thermal infrared dataset [J]. Remote Sensing of Environment, 2023, 298: 113840.), a large number of statistical analyses of the radiation brightness characteristics of civil aircraft in the Sustainable Development Science Satellite 1 (SDGSAT-1) infrared image are conducted, and all of them are lower than the background (land, ocean) where the civil aircraft target is located. In the infrared band, the radiation brightness of ground objects is mainly related to the temperature of the ground objects themselves. The skin temperature of civil aircraft targets is generally between -25 and -30 degrees Celsius when cruising at high altitudes (Fehrm B. Bjorn's Corner: Supersonic transport revival, Part 6 [EB / OL]. (2018-09-14) [2024-04-29]. https: / / leehamnews.com / 2018 / 09 / 14 / bjorns-corner-supersonic-transport-revival-part-6 / ), which is generally lower than the background of land, ocean and other surfaces. For the simulation of civil aircraft targets in infrared images, this method takes the average value of the radiation brightness values ​​of the first 30% of the highest pixels in the infrared image, V I As the background radiation brightness value, the reference radiation of all pixels of the target is calculated as rV I , r is the scaling factor, and its value is in [0,1].

[0044] S7. Based on the linear mixed model, the target full pixel radiation is injected into the satellite optical image according to the target abundance matrix to generate a simulated target.

[0045] According to the linear mixture model, the target grayscale is injected into the image: V′ i,j =(1-A i,j )V i,j +A i,j (1-r)V I ;

[0046] Among them, V i,j is the gray value of the pixel (i, j) on the original infrared image, V′ i,j is the pixel grayscale value after adding the simulation target.

[0047] Two real civil aircraft targets are cropped from the public SDGSAT-1 satellite infrared image, and their imaging information is shown in Table 1. The two targets are numbered as Target 1 and Target 2. The method of this embodiment is used to simulate and reproduce the two measured civil aircraft targets, and the simulation parameters are shown in Table 2.

[0048] Table 1 Publicly available imaging information of civil aircraft targets measured by SDGSAT-1 satellite

[0049]

[0050] Table 2 Publicly available SDGSAT-1 satellite measured airborne civil aircraft target simulation and reproduction parameter settings

[0051]

[0052]

[0053] Figure 3 (a) and Figure 4 In (a), 30×30 images around target 1 and target 2 are cropped respectively, and the two targets are simulated and reproduced according to the simulation parameters in Table 4. The blue frame is the measured civil aircraft target, and the red frame is the simulated civil aircraft target. Except for the contrail, the simulated civil aircraft target basically reproduces the grayscale features and morphological features of the real civil aircraft target, which verifies the effectiveness of the simulation method of this embodiment.

[0054] 1000 images of 256×256 size were cropped from the public SDGSAT-1 satellite infrared images. Each image was used as a base image to generate an infrared image sequence. 3-10 simulated civil aircraft targets were added to each sequence, generating a total of 1000 simulated infrared image sequences. Each sequence contained 50 frames of simulated images with the same background base.

[0055] The target detection algorithm is trained using the simulation data as its training set, and the trained target detection algorithm is directly used to detect real civil aircraft in the public SDGSAT-1 satellite infrared images. Figure 5 The detection results of four real civil aircraft on the public infrared images of three bands of SDGSAT-1 satellite are given by the detection algorithm trained with simulation data. Figure 5 In the figure, the green box indicates the correctly detected target, the yellow box indicates the undetected target, and the red box indicates the false alarm reported by the detection algorithm. Figure 5 It can be seen that the detection algorithm trained based on simulation data has better detection performance for real civil aircraft. 11 out of 12 targets can be detected, with only 1 missed detection and 4 false alarms, verifying the effectiveness of the simulation method of this embodiment in constructing a satellite image weak moving target data set.

[0056] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0057] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A satellite optical image weak moving target simulation method, characterized in that: The following steps are involved: S1. Input a single satellite optical image Among them, H represents the image height, W represents the image width, and C represents the number of image channels; S2, setting the simulation target speed, motion trajectory, and target starting point, and obtaining the coordinate value and heading angle of the target center point on the two-dimensional image plane; S3, approximate and simplify the two-dimensional image plane shape of the modeling target to obtain an approximate shape model of the target; S4. According to the target center point coordinate value and target heading angle, the target approximate shape model is embedded into the image and the target abundance matrix is ​​calculated. S5, performing Gaussian blur processing on the target abundance matrix to simulate the point diffusion phenomenon in the imaging process, and obtaining the target abundance matrix after Gaussian blur; S6. Inject the target full pixel radiation into the satellite optical image according to the target abundance matrix after Gaussian blur to generate a simulated target.

2. The satellite optical image weak moving target simulation method according to claim 1, characterized in that: Target abundance A at pixel (i, j) i,j The calculation formula is: Among them, T i,j is the area occupied by the target approximate shape model on pixel (i, j), and P is the area of ​​a single pixel.

3. The satellite optical image weak moving target simulation method according to claim 1 or 2, characterized in that: The pixel gray value V of the simulation target i, ' j The expression is as follows: In i,j =(1-A i,j )V i,j +A i,j (1-r)V I ,; Among them, V i,j is the radiance or radiance curve at pixel (i, j) on the original satellite optical image, A i,j is the target abundance after Gaussian blur at pixel (i, j), V I is the average radiation brightness value or average radiation brightness curve of the top M% pixels in the original infrared image, rV I is the reference radiation brightness value or radiation brightness curve of all target pixels, and r is the set scale factor.

4. The satellite optical image weak moving target simulation method according to claim 3, characterized in that: The value of r is [0,1].

5. The satellite optical image weak moving target simulation method according to claim 3, characterized in that: M＝30。

Images