Fusion method of simulation target and background image in injection type simulation system

Abstract

The invention discloses a fusion method of a simulation target and a background in an injection type simulation system. Firstly, interpolation up-sampling is carried out on a part of background images, and details of a target and a background are increased; then, the maximum value of the overlapped part of the target and the background is obtained, and fusion of the target and the background is achieved; and finally, carrying out nearest neighbor downsampling on the fused image to obtain an image with the injected resolution. According to the method, the fusion effect can be better by completing the image fusion of the target and the background in the up-sampling image, and the visual authenticity of the fusion image of the injection type simulation system is effectively improved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of photoelectric simulation, and mainly relates to a simulation target and background image fusion method of an injection type simulation system. The injection type simulation system needs to fuse the simulation target with the background image collected in the field according to the position information of the target. The smooth transition of the target and the background edge is one of the scene reality evaluation indexes of the simulation image. The relatively real simulation target and the fused image also help to improve the effectiveness of the target detection and recognition algorithm. BACKGROUND

[0002] The rapid development of modern air targets brings new challenges to photoelectric detection technology, and the infrared search and tracking system needs to have better detection performance. However, the update of the system is realized by continuously accumulating experience through field tests or actual combat exercises. The injection type simulation system can reduce the cost of manpower and material resources and the loss of the detector in the field test, can save the time cost of the development of the target detection and recognition algorithm, provides a platform guarantee for the continuous upgrading of the algorithm, and provides technical support for the development of the infrared search and tracking system, so it is particularly important to deeply study the injection type simulation system.

[0003] The injection type simulation system needs to generate a simulation target in the current state according to the optical parameters, electrical parameters, distance between the target and the device, and flight attitude of the target, and fuse the simulation target with the measured background image to generate a relatively real simulation scene. The fusion effect of the target and the background directly affects the effectiveness of the target detection and recognition algorithm, so it becomes a difficult task to study the fusion method between the simulation target and the measured background. However, due to the mutual occlusion of the target and the background and the mutual influence of the infrared radiation in the actual scene, the image fusion between the simulation target and the measured background is relatively complex.

[0004] Image fusion has three levels, namely pixel level, decision level and feature level. Among the three fusion levels, the pixel level image fusion method has strong ability to retain high-frequency information of the image, so that the image has more rich, accurate and reliable detailed information. The injection type simulation system needs pixel level fused images. The pixel level image fusion methods include weighted average method, pixel gray value maximum / minimum method, Poisson fusion method and wavelet transform method, etc. Among them, the weighted average method and the gray maximum fusion method are basically based on isolated pixel fusion, and the sudden change of the fusion boundary is not considered. The Poisson fusion method and the wavelet transform method have large amount of calculation, and it is difficult to apply to the real-time infrared search and tracking system injection type simulation engineering.

[0005] In summary, the current target and background fusion algorithm has the problems of poor real-time performance and non-smooth transition of fusion edge. The reason for poor real-time performance is that the algorithm is relatively complex, and with the increase of the resolution of the detector, the complexity of the calculation is greatly increased. Therefore, the multi-scale based image fusion method proposed in the present application has important significance for enhancing the visual reality of the injection simulation system. SUMMARY

[0006] In order to overcome the shortcomings of the prior art, the present application provides a fusion method for simulating target and background images in an injection simulation system, which is a target and background fusion method with good edge fusion effect and good real-time performance, to solve the problems of low fusion timeliness and poor edge effect in the prior art.

[0007] The difficulty of simulating target and background image fusion lies in the pixel value of the overlapping part, especially the processing of the image edge. The edge is prone to sawtooth shape (see Figure 3 ), which affects the visual reality of the simulation scene. The method first interpolates the partial background image for upsampling to increase the details of the target and the background. Then, the maximum value is taken for the overlapping part of the target and the background to realize the fusion of the target and the background. Finally, the fused image is nearest neighbor down-sampled to obtain the injection resolution scene image. Completing image fusion in a high scale image (i.e. a high resolution image) can make the fusion effect of the target and the background better, and the simulation target is generated based on the real parameters of the device and the real flight parameters of the target, avoiding the influence caused by the inconsistency of the gray levels of the two images in image fusion.

[0008] The above object of the present application is achieved by the following technical scheme:

[0009] A fusion method for simulating target and background images in an injection simulation system, the method comprising the following steps:

[0010] (1) Partially up-sample the infrared 360° background image to generate a high-resolution background area image.

[0011] In order to improve the real-time performance and applicability of the injection simulation system, when imaging simulation is performed on the simulation target, a high-resolution simulation target infrared image is first generated, the simulation instantaneous field of view is 0.06 mrad, and then the infrared 360° background image in which the simulation target is located is partially up-sampled. The resolution of the up-sampled background area image is the resolution of the simulation target infrared image.

[0012] According to the azimuth angle and the elevation angle of the simulation target, the pixel position of the simulation target in the infrared 360° background image is first located, and the background area corresponding to the simulation target is selected for partial up-sampling according to the size of the simulation target area. The pixel index of the background area where the simulation target is located is:

[0013]

[0014] where bx, by are pixel indices in x and y directions respectively, azi is the azimuth angle of the simulated target, col is the number of columns of the infrared 360° background image, pit is the pitch angle of the simulated target, pitB is the device pitch baseline corresponding to the background image, fov is the field of view of the device, row is the number of rows of the background image.

[0015] The size of the background region is:

[0016]

[0017] where bsx, bsy are the number of pixels of the background region corresponding to the simulated target in x and y directions respectively, tx and ty are the number of pixels of the simulated target in x and y directions respectively, and M is the up-sampling frequency of the local background image.

[0018] The pixel of the up-sampled background region is:

[0019] f b1 (x·(1,2,…,M),y·(1,2,…,M))=f b2 (x,y)

[0020]

[0021] where f b1 is the pixel of the up-sampled background region, f b2 is the pixel of the original background region, x1 and x2 are the minimum index and the maximum index of the x direction pixel of the simulated target after local up-sampling respectively, and y1 and y2 are the minimum index and the maximum index of the y direction pixel of the simulated target after local up-sampling respectively.

[0022] The high-resolution scene image is generated by fusing the high-resolution local up-sampled simulated target image and the high-resolution background region image based on the maximum value method:

[0023] f bt (x,y)=max(f t1 (x,y),f b1 (x,y))

[0024] where f b1 is the pixel of the up-sampled background region, f t1 is the pixel of the simulated target image.

[0025] The high-resolution scene image is down-sampled by the nearest neighbor method to obtain the injected resolution scene image:

[0026] f(x,y)=fbt (Mx,My)

[0027] In the formula, M is a down-sampling frequency, and in order to obtain an image of original resolution, the down-sampling frequency is the same as the up-sampling frequency of the background image.

[0028] In order to adapt to different specifications of infrared detectors, when simulating the simulation target, a high-resolution infrared image is first generated, and then the up-sampling frequency of the local background image and the down-sampling frequency of the fused image are determined according to the instantaneous field of view of the actual infrared detector and the instantaneous field of view of the simulation image, which is generally the multiple of the instantaneous field of view of the 360° background region relative to the instantaneous field of view of the simulation target.

[0029] Compared with the prior art, the beneficial effects of the present application are:

[0030] 1) The up-sampling operation of the simulation target makes the simulation target simulation algorithm applicable to simulation of infrared detectors of various specifications;

[0031] 2) Multi-scale image fusion makes the fusion edge of the simulation target and the measured background image smooth transition, and the whole is natural and realistic;

[0032] 3) The calculation method is relatively simple, and the real-time performance of the injection simulation system can be improved. BRIEF DESCRIPTION OF DRAWINGS

[0033] Figure 1 is the flowchart for the implementation of the present application.

[0034] Figure 2 is a fusion local image of the tail part of the plane of the simulation target and the measured background in specific embodiment 1, and from the figure, it can be seen that the edge fusion effect of the target and the background is better, and is more in line with the measured scene;

[0035] Figure 3 is a fusion local image of the tail part of the plane of the simulation target and the measured background in the prior art, and from the figure, it can be seen that the edge fusion of the target and the background has obvious sawtooth shape, and the fusion effect is relatively poor. DETAILED DESCRIPTION

[0036] The technical solutions in the implementation of the present application will be described in detail below in combination with the drawings in the implementation of the present application. Some parameters are involved, and these parameters need to be adjusted according to the specific processing environment to achieve good performance.

[0037] Referring to Figures 1 to 3 , specific embodiment 1, the embodiment 1 provides a fusion method of simulation target and background image in an injection simulation system, and the simulation environment is Visual Studio 2019

[0038] Simulation method: based on the light projection algorithm to establish the target simulation platform, the Boeing 747 is simulated, the distance between the target and the photoelectric device is 10km, the instantaneous field of view of the device is 0.18mrad, the instantaneous field of view of the simulation target is 0.06mrad, the simulation target and the device measured background image are fused.

[0039] Implementation steps: (1) the local up-sampling of the infrared 360° background image is carried out, and a high-resolution background area image is generated.

[0040] In order to improve the real-time performance and applicability of the injection simulation system, when the simulation target is imaged, a high-resolution simulation target infrared image is first generated, the simulation instantaneous field of view is 0.06mrad, and then the infrared 360° background image where the simulation target is located is locally up-sampled, and the resolution of the sampled background area image is the resolution of the simulation target infrared image.

[0041] According to the azimuth and elevation angle of the simulation target, the pixel position of the simulation target in the infrared 360° background image is first located, and the background area corresponding to the simulation target is selected for local up-sampling according to the area size of the simulation target. The pixel index of the background area where the simulation target is located is:

[0042]

[0043] In the formula, bx and by are the pixel indexes in the x direction and the y direction respectively, azi is the azimuth of the simulation target, col is the number of columns contained in the infrared 360° background image, pit is the elevation angle of the simulation target, pitB is the device elevation baseline corresponding to the background image, fov is the field of view of the device, and row is the number of rows contained in the background image.

[0044] The size of the background area is:

[0045]

[0046] In the formula, bsx and bsy are the number of pixels in the x direction and the y direction of the background area corresponding to the simulation target respectively, tx and ty are the number of pixels of the simulation target in the x direction and the y direction respectively, and M is the up-sampling frequency of the local background image.

[0047] The pixel of the up-sampled background area is:

[0048] f b1 (x·(1,2,…,M),y·(1,2,…,M))=f b2 (x,y)

[0049]

[0050] In the formula, f b1is the background area pixel after upsampling, f b2 is the original background area pixel, x1 and x2 are the minimum index and maximum index of the x-direction pixel after local upsampling of the simulation target, and y1 and y2 are the minimum index and maximum index of the y-direction pixel after local upsampling of the simulation target.

[0051] Based on the maximum value method, the high-resolution local up-sampled simulation target image is fused with the high-resolution background area image to generate a high-resolution scene image:

[0052] f bt (x,y)=max(f t1 (x,y),f b1 (x,y))

[0053] Where, f b1 is the background area pixel after upsampling, f t1 is the simulated target image pixel.

[0054] Perform nearest neighbor downsampling on the high-resolution scene image to obtain the injected resolution scene image:

[0055] f(x,y)=f bt (Mx,My)

[0056] Where M is the downsampling frequency. In order to obtain the image with the original resolution, the downsampling frequency here is the same as the upsampling frequency of the background image.

[0057] In order to adapt to infrared detectors of different specifications, a high-resolution infrared image is first generated when simulating the simulated target. Then, the upsampling frequency of the local background image and the downsampling frequency of the fused image are determined based on the instantaneous field of view of the actual infrared detector and the instantaneous field of view of the simulated image. Generally, it is a multiple of the instantaneous field of view of the 360° background area relative to the instantaneous field of view of the simulated target.

[0058] In this embodiment 1, the above method is adopted and the following parameters are selected to achieve the fusion of the simulation target and the background image in the injection simulation system:

[0059] The total number of rows (row) and the total number of columns (col) of the background image with a 360° azimuth and a 10° pitch is 512 and 30,000, respectively.

[0060] The instantaneous field of view of the device is 0.18mrad, and the instantaneous field of view of the simulated target is 0.06mrad. Therefore, the actual target size in the device should be 1 / 9 of the simulated target, and M is 3;

[0061] Target azimuth angle: 30°

[0062] The target pitch angle pit is: 4° The pitch baseline pitB corresponding to the background image is: 0°

[0063] The calculated x-direction starting index bx of the simulation target in the background image is: 2500

[0064] The starting index in the y direction is: 205;

[0065] For a Boeing 747 with a length of 76.3 meters and a wingspan of 68.4 meters, the imaging size of the fuselage length is 126 pixels and the imaging size of the wingspan length is 114 pixels at a distance of 10 km in a simulation field of view with an instantaneous field of view of 0.06 mrad.

[0066] Then the size of the corresponding background area image is:

[0067] Bsx is: 126 / M=42; bsy is: 114 / M=38

[0068] The local background area of ​​Bsx*bsy is upsampled by 3*3, and the maximum image is fused with the simulation target to obtain a 126*114 fused image. Then it is downsampled by 3*3 to obtain a 42*38 fused image corresponding to the instantaneous field of view of the 0.18mrad device. Figure 2 ,The fusion local image of the tail of the aircraft with the simulated target and the measured background, from the figure we can see that the edge fusion effect of the target and the background is better, the grayscale is smoother, and it is more consistent with the measured scene. Figure 3 ,from Figure 3 It can be seen that the edge fusion of the target and the background has obvious jagged edges, and the fusion effect is relatively poor.

Claims

1. A method for fusing a simulation target and a background image in an injection-type simulation system, characterized by: The fusion method comprises the following steps: (1) Locally upsample the infrared 360° background image to generate a high-resolution background area image; (2) Based on the maximum value method, the high-resolution simulation target image and the high-resolution background area image are fused to generate a high-resolution scene image; (3) Perform nearest neighbor downsampling on the high-resolution scene image to obtain the injected resolution scene image.

2. The method for fusing a simulation target and a background image in an injection-type simulation system according to claim 1, characterized in that: In step (1), when performing imaging simulation on the simulated target, a high-resolution infrared image of the simulated target is first generated, and the simulated instantaneous field of view is 0.06 mrad. Then, the infrared 360° background image where the simulated target is located is locally upsampled, and the resolution of the sampled background area image is the resolution of the infrared image of the simulated target.

3. The method for fusing a simulation target and a background image in an injection-type simulation system according to claim 2, characterized in that: In step (1), the pixel position of the simulated target in the infrared 360° background image is first located according to the azimuth and elevation angles of the simulated target. The background area corresponding to the simulated target is selected for local upsampling according to the area size of the simulated target. The pixel index of the background area where the simulated target is located is: Where bx and by are the pixel indices in the x and y directions respectively, azi is the azimuth of the simulated target, col is the number of columns in the infrared 360° background image, pit is the pitch angle of the simulated target, pitB is the device pitch baseline corresponding to the background image, fov is the field of view of the device, and row is the number of rows in the background image. The background area size is: Where bsx and bsy are the number of pixels in the background area corresponding to the simulated target in the x and y directions, tx and ty are the number of pixels in the background area corresponding to the simulated target in the x and y directions, and M is the upsampling frequency of the local background image. The upsampled background area pixels are: Where, f b1 is the background area pixel after upsampling, f b2 is the original background area pixel, x1 and x2 are the minimum index and maximum index of the x-direction pixel after local upsampling of the simulation target, and y1 and y2 are the minimum index and maximum index of the y-direction pixel after local upsampling of the simulation target.

4. The method for fusing a simulation target and a background image in an injection-type simulation system according to claim 3, wherein: In step (2), a high-resolution scene image is generated after fusion: f bt (x,y)=max(f t1 (x,y),f b1 (x,y)) Where, f b1 is the background area pixel after upsampling, f t1 is the simulated target image pixel.

5. The method for fusing a simulation target and a background image in an injection-type simulation system according to claim 4, characterized in that: In step (3), the high-resolution scene image is downsampled by the nearest neighbor to obtain the injected resolution scene image: f(x,y)=f bt (Mx,My) Where M is the downsampling frequency. In order to obtain the image with the original resolution, the downsampling frequency here is the same as the upsampling frequency of the background image.

6. The method for fusing a simulation target and a background image in an injection-type simulation system according to claim 5, characterized in that: In order to adapt to infrared detectors of different specifications, a high-resolution infrared image is first generated when simulating the simulated target. Then, the upsampling frequency of the local background image and the downsampling frequency of the fused image are determined according to the instantaneous field of view of the actual infrared detector and the instantaneous field of view of the simulated image, which is a multiple of the instantaneous field of view of the 360° background area relative to the instantaneous field of view of the simulated target.

Images