A method for fusing simulation targets and background images in an injection simulation system

Abstract

The application discloses a kind of injection simulation system in the fusion method of simulation target and background. First, interpolation upsampling is carried out on part of background image, to increase the details of target and background;Then the maximum value is taken to the overlapping part of target and background, to realize the fusion of target and background;Finally, the fused image is nearest neighbor down-sampled, and the image of injection resolution is obtained. This method can make the fusion effect better by completing the image fusion of target and background in the up-sampled image, effectively improving the visual reality of the fused image of injection simulation system.

Description

Technical fields:

[0001] This invention belongs to the field of optoelectronic simulation technology, and its main content is a method for fusing simulated target and background images in an injection-type simulation system. The injection-type simulation system needs to fuse the simulated target with an externally acquired background image based on the target's location information. Smooth transition between the target and background edges is one of the evaluation indicators of the scene realism of the simulated image. A more realistic simulated target and fused image also help improve the effectiveness of target detection and recognition algorithms. Background technology:

[0002] The rapid development of modern aerial targets has brought new challenges to photoelectric detection technology, requiring infrared search and track systems to have better detection performance. However, system upgrades are achieved through continuous field testing and practical exercises to accumulate experience. Injection-based simulation systems can reduce the manpower and material costs of field testing and the wear and tear on detectors, save time and costs in developing target detection and recognition algorithms, provide a platform for continuous algorithm upgrades, and provide technical support for the development of infrared search and track systems. Therefore, in-depth research on injection-based simulation systems is particularly important.

[0003] Injection-based simulation systems require generating a simulated target in its current state based on information such as the device's optical and electrical parameters, the distance between the target and the device, and the target's flight attitude. The simulated target is then fused with a measured background image to generate a relatively realistic simulation scene. The fusion effect directly impacts the effectiveness of the target detection and recognition algorithm, making the research into fusion methods between simulated targets and measured backgrounds a challenging task. However, due to mutual occlusion and infrared radiation interference between targets and backgrounds in real-world scenarios, image fusion between simulated targets and measured backgrounds is quite complex.

[0004] Image fusion has three levels: pixel-level, decision-level, and feature-level. Among these three levels, pixel-level image fusion methods have a strong ability to retain high-frequency information, resulting in richer, more accurate, and more reliable detail information. Injection simulation systems require pixel-level fused images. Pixel-level image fusion methods include weighted averaging, pixel grayscale value maximization / minimum fusion, Poisson fusion, and wavelet transform, among others. However, weighted averaging and grayscale maximum fusion methods are primarily based on fusion of isolated pixels and lack consideration for abrupt changes in the fusion boundary. Poisson fusion and wavelet transform methods have high computational costs, making them difficult to apply in real-time infrared search and tracking system injection simulation engineering.

[0005] In summary, current target-background fusion algorithms suffer from poor real-time performance and uneven transitions at fusion edges. The poor real-time performance is due to the complexity of the algorithms, which increases significantly with detector resolution. Therefore, the multi-scale image fusion method proposed in this invention is of great significance for enhancing the visual realism of injection-based simulation systems. Summary of the Invention:

[0006] To overcome the shortcomings of existing technologies, this invention provides a method for fusing simulated target and background images in an injection simulation system. This method is a target-background fusion method with good edge fusion effect and good real-time performance, thereby solving the problems of low fusion efficiency and poor edge effect in existing technologies.

[0007] The difficulty in fusing the simulated target and background images lies in the pixel values ​​of the overlapping areas, especially the processing of image edges, which are prone to jagged edges (see...). Figure 3 This method addresses the issue of inconsistent grayscale levels in simulated scenes. First, it interpolates and upsamples a portion of the background image to enhance the details of both the target and background. Then, it maximizes the value of the overlapping area between the target and background to achieve fusion. Finally, it performs nearest-neighbor downsampling on the fused image to obtain the injected resolution scene image. Performing image fusion in a high-scale (i.e., high-resolution) image improves the fusion effect between the target and background. Furthermore, since the simulated target is generated based on the actual parameters of the equipment and the target's actual flight parameters, it avoids the impact of inconsistent grayscale levels between the two images during image fusion.

[0008] The above-mentioned objective of the present invention is achieved through the following technical solution:

[0009] A method for fusing simulated target and background images in an injection-based simulation system, the method comprising the following steps:

[0010] (1) Local upsampling of the infrared 360° background image to generate a high-resolution background region image.

[0011] To improve the real-time performance and applicability of the injection simulation system, a high-resolution infrared image of the simulated target is first generated when performing imaging simulation on the simulated target. The instantaneous field of view of the simulation is 0.06 mrad. Then, the infrared 360° background image where the simulated target is located is locally upsampled, and the resolution of the sampled background area image is the same as the resolution of the infrared image of the simulated target.

[0012] First, the pixel position of the simulated target in the 360° infrared background image is located based on its azimuth and elevation angles. Then, the background region corresponding to the simulated target is selected for local upsampling based on the target's area. The pixel index of the background region containing the simulated target is:

[0013]

[0014] In the formula, bx and by are the pixel indices in the x and y directions, respectively; azi is the azimuth angle of the simulated target; col is the number of columns in the infrared 360° background image; pit is the pitch angle of the simulated target; pitB is the device pitch baseline corresponding to the background image; fov is the field of view of the device; and row is the number of rows in the background image.

[0015] The background area size is:

[0016]

[0017] In the formula, bsx and bsy are the number of pixels in the background region corresponding to the simulated target in the x and y directions, respectively; tx and ty are the number of pixels in the simulated target in the x and y directions, respectively; and M is the upsampling frequency of the local background image.

[0018] The upsampled background region pixels are:

[0019] f b1 (x·(1,2,…,M),y·(1,2,…,M))=f b2 (x,y)

[0020]

[0021] In the formula, f b1 f represents the upsampled background region pixels. b2 y1 and y2 are the minimum and maximum indices of the pixels in the x-direction after local upsampling of the simulated target, respectively, and the minimum and maximum indices of the pixels in the y-direction after local upsampling of the simulated target, respectively.

[0022] A high-resolution scene image is generated by fusing a high-resolution locally upsampled simulated target image with a high-resolution background region image using the maximum value method.

[0023] f bt (x,y)=max(f t1 (x,y),f b1 (x,y))

[0024] In the formula, f b1 f represents the upsampled background region pixels. t1 To simulate the pixels of the target image.

[0025] Nearest neighbor downsampling is performed on the high-resolution scene image to obtain the injected resolution scene image:

[0026] f(x,y)=fbt (Mx,My)

[0027] In the formula, M is the downsampling frequency. In order to obtain the image at the original resolution, the downsampling frequency here is the same as the upsampling frequency of the background image.

[0028] To accommodate infrared detectors of different specifications, a high-resolution infrared image is first generated when simulating the target. Then, the upsampling frequency of the local background image and the downsampling frequency of the fused image are determined based on the instantaneous field of view of the actual infrared detector and the instantaneous field of view of the simulated image. Generally, it is the multiple of the instantaneous field of view of the 360° background region relative to the instantaneous field of view of the simulated target.

[0029] Compared with the prior art, the beneficial effects of the present invention are:

[0030] 1) The upsampling operation on the simulated target makes the simulation algorithm applicable to the simulation of infrared detectors of various specifications;

[0031] 2) Multi-scale image fusion enables a smooth transition at the fusion edges between the simulated target and the measured background image, resulting in a natural and realistic overall appearance;

[0032] 3) The calculation method is relatively simple, which makes it easier to improve the real-time performance of the injection simulation system. Attached Figure Description

[0033] Figure 1 This is a flowchart illustrating the implementation of the present invention.

[0034] Figure 2 The image shows a partial view of the aircraft tail section where the simulated target and the actual background are fused in Specific Implementation Example 1. It can be seen from the image that the edge fusion effect between the target and the background is better and more consistent with the actual test scenario.

[0035] Figure 3 The image shows a partial fusion of the simulated target and the measured background of an aircraft tail using existing technology. It can be seen from the image that the edges of the target and the background have obvious jaggedness, and the fusion effect is relatively poor. Detailed Implementation

[0036] The technical solution of this invention will now be described in detail with reference to the accompanying drawings. Several parameters are involved, and these parameters need to be adjusted for specific processing environments to achieve optimal performance.

[0037] See Figures 1 to 3 Specific embodiment 1: This embodiment 1 provides a method for fusing simulation target and background images in an injection-based simulation system. Simulation environment: Visual Studio 2019.

[0038] Simulation method: A target simulation platform was established based on the ray casting algorithm. The Boeing 747 was simulated. The distance between the target and the optoelectronic equipment was 10km. The instantaneous field of view of the equipment was 0.18mrad. The instantaneous field of view of the simulated target was 0.06mrad. The simulated target and the measured background image of the equipment were fused together.

[0039] Implementation steps: (1) Local upsampling of the infrared 360° background image to generate a high-resolution background area image.

[0040] To improve the real-time performance and applicability of the injection simulation system, a high-resolution infrared image of the simulated target is first generated when performing imaging simulation on the simulated target. The instantaneous field of view of the simulation is 0.06 mrad. Then, the infrared 360° background image where the simulated target is located is locally upsampled, and the resolution of the sampled background area image is the same as the resolution of the infrared image of the simulated target.

[0041] First, the pixel position of the simulated target in the 360° infrared background image is located based on its azimuth and elevation angles. Then, the background region corresponding to the simulated target is selected for local upsampling based on the target's area. The pixel index of the background region containing the simulated target is:

[0042]

[0043] In the formula, bx and by are the pixel indices in the x and y directions, respectively; azi is the azimuth angle of the simulated target; col is the number of columns in the infrared 360° background image; pit is the pitch angle of the simulated target; pitB is the device pitch baseline corresponding to the background image; fov is the field of view of the device; and row is the number of rows in the background image.

[0044] The background area size is:

[0045]

[0046] In the formula, bsx and bsy are the number of pixels in the background region corresponding to the simulated target in the x and y directions, respectively; tx and ty are the number of pixels in the simulated target in the x and y directions, respectively; and M is the upsampling frequency of the local background image.

[0047] The upsampled background region pixels are:

[0048] f b1 (x·(1,2,…,M),y·(1,2,…,M))=f b2 (x,y)

[0049]

[0050] In the formula, f b1f represents the upsampled background region pixels. b2 y1 and y2 are the minimum and maximum indices of the pixels in the x-direction after local upsampling of the simulated target, respectively, and the minimum and maximum indices of the pixels in the y-direction after local upsampling of the simulated target, respectively.

[0051] A high-resolution scene image is generated by fusing a high-resolution locally upsampled simulated target image with a high-resolution background region image using the maximum value method.

[0052] f bt (x,y)=max(f t1 (x,y),f b1 (x,y))

[0053] In the formula, f b1 f represents the upsampled background region pixels. t1 To simulate the pixels of the target image.

[0054] Nearest neighbor downsampling is performed on the high-resolution scene image to obtain the injected resolution scene image:

[0055] f(x,y)=f bt (Mx,My)

[0056] In the formula, M is the downsampling frequency. In order to obtain the image at the original resolution, the downsampling frequency here is the same as the upsampling frequency of the background image.

[0057] To accommodate infrared detectors of different specifications, a high-resolution infrared image is first generated when simulating the target. Then, the upsampling frequency of the local background image and the downsampling frequency of the fused image are determined based on the instantaneous field of view of the actual infrared detector and the instantaneous field of view of the simulated image. Generally, it is the multiple of the instantaneous field of view of the 360° background region relative to the instantaneous field of view of the simulated target.

[0058] In this embodiment 1, the above method is used, and the following parameters are selected to achieve the fusion of the simulation target and the background image in the injection simulation system:

[0059] The background image has a total of 512 rows and 30,000 columns, with an azimuth of 360° and a tilt of 10°.

[0060] The instantaneous field of view of the device is 0.18 mrad, and the instantaneous field of view of the simulated target is 0.06 mrad. Therefore, the actual size of the target in the device should be 1 / 9 of the size of the simulated target, and M is 3.

[0061] The target azimuth angle (azi) is 30°.

[0062] The target pitch angle (pit) is 4°, and the pitch baseline (pitB) corresponding to the background image is 0°.

[0063] The calculated starting index bx of the simulated target in the background image in the x-direction is 2500.

[0064] The starting index by in the y-direction is 205;

[0065] For a Boeing 747, which is 76.3 meters long and has a wingspan of 68.4 meters, in a simulated field of view with an instantaneous field of view of 0.06 mrad, the imaging size of the fuselage length at a distance of 10 km is 126 pixels, and the imaging size of the wingspan length is 114 pixels.

[0066] Therefore, the size of the corresponding background region in the image is:

[0067] Bsx is 126 / M = 42; bsy is 114 / M = 38

[0068] The local background region of Bsx*bsy is upsampled by 3*3 and fused with the simulated target at its maximum value to obtain a fused image of 126*114. Then, it is downsampled by 3*3 to obtain a fused image of 42*38 corresponding to the instantaneous field of view of the 0.18mrad device. See [link / reference]. Figure 2 The image shows a fused partial view of the aircraft tail between the simulated target and the measured background. It can be seen that the edge blending effect between the target and the background is better, the grayscale is smoother, and it better matches the measured scene. In contrast, a fused partial view of the aircraft tail using existing technology is shown below. Figure 3 ,from Figure 3 As can be seen, the edges of the target and the background are noticeably jagged, indicating a poor blending effect.

Claims

1. A method for fusing simulated target and background images in an injection-type simulation system, characterized in that: The fusion method includes the following steps: (1) For infrared The background image is locally upsampled to generate a high-resolution background region image; in step (1), the simulated target is first located in the infrared region based on the azimuth and elevation angles of the simulated target. The pixel positions in the background image are selected by upsampling the background region corresponding to the simulated target area based on the size of the simulated target. The pixel index of the background region where the simulated target is located is: In the formula, , They are respectively direction and Pixel index of direction, The azimuth angle of the simulated target. For infrared The number of columns contained in the background image. The pitch angle of the simulated target. The device pitch baseline corresponding to the background image. The size of the device's field of view. The number of rows contained in the background image; The background area size is: In the formula, , The background area corresponding to the simulation target is in direction and Number of pixels in the direction, and The simulation target is in direction and The number of pixels in the direction, where M is the upsampling frequency of the local background image; The upsampled background region pixels are: In the formula, These are the upsampled background region pixels. These are the original background area pixels. Local upsampling of the simulated target The minimum and maximum indices of the directional pixels. Local upsampling of the simulated target The minimum and maximum indices of the directional pixels; (2) Based on the maximum value method, the high-resolution simulation target image and the high-resolution background region image are fused to generate a high-resolution scene image; (3) Perform nearest neighbor downsampling on the high-resolution scene image to obtain the injected resolution scene image; To accommodate infrared detectors of different specifications, a high-resolution infrared image is first generated when simulating the target. Then, based on the instantaneous field of view of the actual infrared detector and the instantaneous field of view of the simulated image, the upsampling frequency of the local background image and the downsampling frequency of the fused image are determined, which is the multiple of the instantaneous field of view of the 360° background region relative to the instantaneous field of view of the simulated target.

2. The method for fusing simulation target and background images in an injection-type simulation system according to claim 1, characterized in that: In step (1), when performing imaging simulation on the simulated target, a high-resolution infrared image of the simulated target is first generated, with an instantaneous field of view of 0.06 mrad. Then, the infrared image of the simulated target is... The background image is locally upsampled, and the resolution of the sampled background region image is the same as that of the simulated target infrared image.

3. The method for fusing simulation target and background images in an injection simulation system according to claim 2, characterized in that: In step (2), a high-resolution scene image is generated after fusion: In the formula, These are the upsampled background region pixels. To simulate the pixels of the target image.

4. The method for fusing simulation target and background images in an injection simulation system according to claim 3, characterized in that: In step (3), the high-resolution scene image is downsampled to the nearest neighbor to obtain the injected resolution scene image: In the formula, M is the downsampling frequency. In order to obtain the image at the original resolution, the downsampling frequency is the same as the upsampling frequency of the background image.

Images