A method for simulating weak moving targets in satellite optical images

Abstract

The present invention discloses a method for simulating weak moving targets in satellite optical images, which can reproduce simulated targets with a highly accurate morphology of weak targets such as civil aircraft on medium-resolution optical images. Simple means such as two-dimensional image plane shape approximation simplified modeling and target abundance Gaussian blur processing are used to replace the tedious steps of target three-dimensional appearance modeling and atmospheric transmission process modeling in existing simulation software, greatly reducing the difficulty and cost of simulation. The present invention can automatically construct a large-scale satellite image weak moving target dataset, which can be used as a training set for target detection algorithms. It has been verified that the target detection network model trained based on the simulation dataset constructed by this method can effectively detect real civil aircraft targets in satellite images.

Description

Technical Field

[0001] The present invention relates to the field of data simulation, in particular to a satellite optical image weak moving target simulation method. Background Art

[0002] With the continued prosperity of the aviation industry, the importance of real-time monitoring of global civil aircraft activity has become increasingly prominent. Currently, civil aircraft traffic control primarily relies on ground-based radar and the ADS-B system. Ground-based radar used for air traffic control typically operates within a range of only a few hundred kilometers. The ADS-B system passively receives position information transmitted by civil aircraft through ground-based receivers and does not cover most ocean areas. Satellite remote sensing imagery can provide global geographic imagery, and image detection algorithms can be used to determine the number and location of specific land features. Medium-resolution satellite imagery (generally considered to have a ground sampling interval of 5-100 meters) can achieve an imaging swath of hundreds of kilometers, covering a wide range of the Earth's surface in a single scene, making it suitable as a supplementary means for global civil aircraft activity monitoring. However, the size of civil aircraft is similar to the spatial resolution of medium-resolution satellite imagery, making it difficult for the human eye to identify civil aircraft in some medium-resolution satellite imagery. Current data-driven object detection methods, such as Faster RCNN and YOLO, require large training sets with labeled samples. However, the human eye has difficulty identifying civilian aircraft in medium-resolution satellite imagery, and manual object labeling is challenging. Simulation technology can automatically generate simulated images with precise object labels, facilitating the development of algorithms for detecting faint moving targets, such as civilian aircraft, in satellite imagery.

[0003] The Vega series of simulation modules released by Multigen-Paradigm, a German company, is a widely used commercial remote sensing imaging simulation toolkit with relatively complete functions. It can realize dynamic and visual simulation of the entire remote sensing imaging process. The remote sensing imaging simulation software DIRSIG, developed by the Digital Imaging and Remote Sensing (DIRS) Laboratory of the Rochester Institute of Technology, has undergone a series of upgrades and improvements. This software can perform broadband, multispectral, and hyperspectral imaging simulations from visible light to thermal infrared bands, and has been applied to many NASA projects. Simulation research on remote sensing images at home and abroad generally focuses on full-link simulation of the entire remote sensing imaging process. It requires the establishment of multiple models such as ground object scenes, target characteristics, atmospheric transmission, and detector response. The workload required is huge and the theory is complex. In addition, some remote sensing image simulation software has begun to be used commercially, but the procurement cost is relatively high.

[0004] Faint moving target detection technology in satellite imagery can help enhance the ability to monitor global civil aircraft activity using satellite optical imagery and improve current civil aircraft control methods. However, the difficulty of manually labeling faint targets and the high cost of existing remote sensing image simulation software make it difficult to construct datasets for faint moving targets in satellite imagery, hindering the development of this technology.

[0005] The terms used in the present invention are explained as follows:

[0006] Abundance: reflects the coverage of a substance on the surface. The abundance of a substance in a pixel is the proportion of the substance in the pixel.

[0007] Linear mixture model: usually refers to the mixing of two or more signals or substances that follows the principle of linear superposition, that is, the total output or result of the mixture is a linear combination of the individual components. In a linear mixture, the contribution of each component is independent and can be calculated separately and then simply added together. In the field of remote sensing, linear mixture models are often used to explain the signals received by remote sensing sensors. These signals are the superposition of electromagnetic radiation reflected or emitted by different types of surface cover (such as vegetation, soil, water bodies, etc.). The linear mixture model assumes that the signal recorded by the sensor is a weighted average of the signals of various types of surface cover, and the weight is the proportion of each cover type in the sensor's field of view. Summary of the Invention

[0008] The technical problem to be solved by the present invention is to provide a method for simulating weak moving targets in satellite optical images in response to the shortcomings of the existing technology, so as to reproduce the simulated targets with a highly consistent morphology of weak targets such as civil aircraft on medium-resolution optical images.

[0009] To solve the above technical problems, the technical solution adopted by the present invention is: a satellite optical image weak moving target simulation method, comprising the following steps:

[0010] S1. Input a single satellite optical image Among them, H represents the image height, W represents the image width, and C represents the number of image channels;

[0011] S2. Set the simulation target speed, motion trajectory, and target starting point, and obtain the coordinate value and heading angle of the target center point on the two-dimensional image plane;

[0012] S3, approximate and simplify the two-dimensional image plane shape of the modeled target to obtain an approximate shape model of the target;

[0013] S4. According to the target center point coordinate value and target heading angle, the target approximate shape model is embedded in the image and the target abundance matrix is ​​calculated.

[0014] S5, performing Gaussian blur processing on the target abundance matrix to simulate the point diffusion phenomenon in the imaging process, and obtaining the target abundance matrix after Gaussian blur;

[0015] S6. Inject the target full pixel radiation into the satellite optical image according to the target abundance matrix after Gaussian blur to generate a simulated target.

[0016] Target abundance A at pixel (i, j) i,j The calculation formula is:

[0017]

[0018] Among them, T i,j is the area occupied by the target approximate shape model at pixel (i, j), and P is the area of ​​a single pixel.

[0019] Pixel gray value V′ of the simulation target i ′ j The expression is as follows:

[0020] V′ i,j =(1-A i,j )V i,j +A i,j (1-r)V I ,;

[0021] Among them, V i,j is the radiance or radiance curve of pixel (i, j) on the original satellite optical image, A i,j is the target abundance after Gaussian blur at pixel (i, j), V I is the average radiance value or average radiance curve of the top M% pixels in the original infrared image, rV I is the reference radiance value or radiance curve of all target pixels, and r is the set scale factor.

[0022] In the present invention, the value of r is [0, 1].

[0023] In the present invention, M=30.

[0024] Compared with the existing technology, the beneficial effects of the present invention are as follows: the present invention proposes a simple and easy method for simulating weak moving targets in satellite optical images, which can reproduce simulated targets that are highly consistent with the measured morphology of weak targets such as civil aircraft on open source medium spatial resolution satellite images, and can automatically generate a large number of simulated samples and labels of weak moving targets in satellite images, which helps to meet the data requirements for the development of weak moving target detection algorithms in satellite images. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1This is a flow chart of a method according to an embodiment of the present invention;

[0026] Figure 2 Figure 1. Target abundance matrix generation steps for an embodiment of the present invention; (a) target shape modeling, (b) embedding the shape model into an image, (c) target abundance matrix calculation, and (d) Gaussian blurring of the abundance matrix.

[0027] Figure 3 To disclose the measured target 1 and its simulated target; (a) simulated image, (b) real target, (c) simulated target;

[0028] Figure 4 To disclose the measured target 2 and its simulated target; (a) simulated image, (b) real target, (c) simulated target;

[0029] Figure 5 These are the detection results of measured civil aircraft targets in the air; (a) publicly measured target 1, (b) publicly measured target 2, (c) publicly measured target 3, and (d) publicly measured target 4. DETAILED DESCRIPTION

[0030] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0031] The embodiment of the present invention proposes a method for simulating weak moving targets in satellite optical images. The specific process is as follows:

[0032] S1. Single satellite optical image input;

[0033] S2. Set the simulation target speed, motion trajectory, target starting point and other motion characteristic information, and obtain the coordinate value and heading angle of the target center point on the two-dimensional image plane (the simulation single frame image can be directly set);

[0034] S3. Simplified modeling of the target 2D image plane shape approximation;

[0035] Faint targets appear blurred in satellite optical images. Furthermore, satellite imaging of the Earth's surface typically uses nadir observation mode, where the photoelectric sensor directly images the surface area below the satellite. Therefore, this method approximates and simplifies the target's 2D top-down view to construct a 2D image plane shape model. Since the fuselage length and wingspan of current civil aircraft are similar, the sub-satellite observation shape of a civil aircraft is simplified to a cross with a uniform length and width, both of which are l. The outermost side of the cross is set to 0.3 of its length.

[0036] S4. embedding the target approximate shape model into the image and calculating the target abundance matrix;

[0037] According to the target center point coordinates and heading angle, the simplified target shape model is attached to the image. The area ratio of the target shape model on each pixel of the image is calculated, and the target abundance matrix A of the same size as the image is constructed based on this area ratio. The target abundance A at pixel (i, j) is i,j The calculation formula is:

[0038]

[0039] Among them, T i,j is the area occupied by the target approximate shape model at pixel (i, j), and P is the area of ​​a single pixel.

[0040] S5. Gaussian blurring is performed on the target abundance matrix to simulate the nonlinear mixing phenomenon caused by atmospheric transmission.

[0041] Due to phenomena such as light scattering during atmospheric propagation, nonlinear mixing occurs in pixels of remote sensing images. This method uses the Gaussian blur algorithm (https: / / docs.opencv.org / 4.0.0 / d4 / d86 / group__imgproc__filter.html#gaabe8c836e97159a9193fb0b11ac52cf1) to blur the target abundance matrix, effectively replacing the simulation step for nonlinear effects such as atmospheric scattering. The standard deviation parameter of the Gaussian blur kernel is set to σ.

[0042] S6. Setting the target full pixel reference radiance;

[0043] The target's full pixel radiance is set based on the target's spectral radiance characteristics in the satellite image band. Based on the literature (Li L, Zhou X, Hu Z, et al. On-orbit monitoring flying aircraft day and night based on SDGSAT-1 thermal infrared dataset [J]. Remote Sensing of Environment, 2023, 298: 113840.), a statistical analysis of the radiance characteristics of civil aircraft in Sustainable Development Science Satellite-1 (SDGSAT-1) infrared images revealed that the radiance of civil aircraft is lower than the background (land and ocean) where the civil aircraft are located. In the infrared band, the radiant brightness of ground objects is mainly related to the temperature of the ground objects themselves. The skin temperature of civil aircraft targets at high altitudes is generally between -25 and -30 degrees Celsius (Fehrm B. Bjorn's Corner: Supersonic transport revival, Part 6 [EB / OL]. (2018-09-14) [2024-04-29]. https: / / leehamnews.com / 2018 / 09 / 14 / bjorns-corner-supersonic-transport-revival-part-6 / ), which is generally lower than the background of land, ocean, and other surfaces. For the simulation of civil aircraft targets in infrared images, this method takes the average radiant brightness value V of the first 30% of the pixels with the highest radiant brightness in the infrared image. I As the background radiation brightness value, the reference radiation of the target full pixel is calculated as rV I , r is the scaling factor, and its value is in [0,1].

[0044] S7. Based on the linear mixed model, the target full pixel radiation is injected into the satellite optical image according to the target abundance matrix to generate a simulated target.

[0045] According to the linear mixing model, the target grayscale is injected into the image: V′ i,j =(1-A i,j )V i,j +A i,j (1-r)V I ;

[0046] Among them, V i,j is the gray value of the pixel (i, j) on the original infrared image, V′ i,j is the pixel grayscale value after adding the simulation target.

[0047] Two real civilian aircraft targets were cropped from a publicly available SDGSAT-1 satellite infrared image. Their imaging information is shown in Table 1. The two targets are numbered Target 1 and Target 2. The method of this embodiment was used to simulate and reproduce the two measured civilian aircraft targets. The simulation parameters are shown in Table 2.

[0048] Table 1 Publicly available imaging information of civil aircraft targets measured by the SDGSAT-1 satellite

[0049]

[0050] Table 2: Parameter settings for simulation and reproduction of civil aircraft targets measured by the SDGSAT-1 satellite

[0051]

[0052]

[0053] Figure 3 (a) and Figure 4 In (a), a 30×30 pixel image was cropped around Target 1 and Target 2. Simulations were then performed on both targets using the simulation parameters in Table 4. The blue frame represents the measured civil aircraft target, and the red frame represents the simulated civil aircraft target. With the exception of the contrail, the simulated civil aircraft target closely replicates the grayscale and morphological characteristics of the real civil aircraft target, validating the effectiveness of this simulation method.

[0054] 1000 256×256 images were cropped from the public SDGSAT-1 satellite infrared images. Each image was used as a base image to generate an infrared image sequence. 3-10 simulated civil aircraft targets were added to each sequence, generating a total of 1000 simulated infrared image sequences. Each sequence contained 50 frames of simulated images with the same background base.

[0055] The target detection algorithm is trained using the simulation data as its training set, and the trained target detection algorithm is directly used to detect real civil aircraft in the public SDGSAT-1 satellite infrared images. Figure 5 The detection results of four real civil aircraft on the public infrared images of three bands of SDGSAT-1 satellite are given by the detection algorithm trained with simulation data. Figure 5 In the figure, the green box indicates the correctly detected target, the yellow box indicates the undetected target, and the red box indicates the false alarm reported by the detection algorithm. Figure 5 It can be seen that the detection algorithm trained based on simulation data has better detection performance for real civil aircraft. It can detect 11 out of 12 targets, with only 1 missed detection and 4 false alarms, verifying the effectiveness of the simulation method of this embodiment in constructing a dataset of weak moving targets in satellite images.

[0056] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present application.

[0057] Obviously, those skilled in the art may make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.

Claims

1. A satellite optical image weak moving target simulation method, characterized in that: The following steps are involved: S1. Input a single satellite optical image Among them, H represents the image height, W represents the image width, and C represents the number of image channels; S2. Set the simulation target speed, motion trajectory, and target starting point, and obtain the coordinate value and heading angle of the target center point on the two-dimensional image plane; S3, approximate and simplify the two-dimensional image plane shape of the modeled target to obtain an approximate shape model of the target; S4. According to the target center point coordinate value and target heading angle, the target approximate shape model is embedded in the image and the target abundance matrix is ​​calculated. S5, performing Gaussian blur processing on the target abundance matrix to simulate the point diffusion phenomenon in the imaging process, and obtaining the target abundance matrix after Gaussian blur; S6. Injecting the target full pixel radiation into the satellite optical image according to the target abundance matrix after Gaussian blur to generate a simulated target; Pixel gray value V′ of the simulation target i,j The expression is as follows: In i,j =(1-A i,j )V i,j +A i,j (1-r)V I ; Among them, V i,j is the radiance or radiance curve of pixel (i, j) on the original satellite optical image, A i,j is the target abundance after Gaussian blur at pixel (i, j), V I is the average radiance value or average radiance curve of the top M% pixels in the original infrared image, rV I is the reference radiance value or radiance curve of all target pixels, and r is the set scale factor.

2. The satellite optical image weak moving target simulation method according to claim 1, characterized in that: Target abundance A at pixel (i, j) i,j The calculation formula is: Among them, T i,j is the area occupied by the target approximate shape model at pixel (i, j), and P is the area of ​​a single pixel.

3. The satellite optical image weak moving target simulation method according to claim 1, characterized in that: The value of r is [0,1].

4. The satellite optical image weak moving target simulation method according to claim 1, characterized in that: M＝30。

Images