Photoelectric pod image enhancement system and method based on super-resolution

Abstract

The invention relates to the technical field of photoelectric pod image processing, in particular to a super-resolution-based photoelectric pod image enhancement system and method. The photoelectric pod image enhancement method comprises the following steps: capturing low-resolution infrared image data, and outputting the low-resolution infrared image data; receiving low-resolution infrared image data, and performing image quality optimization processing on the infrared image data; carrying out super-resolution processing on the image, and converting low-resolution infrared image data into high-resolution infrared image data; and carrying out visual display on the converted infrared image data. The photoelectric pod image enhancement system comprises a sampling module, an optimization module, an infrared super-resolution module and a display module. By means of the mode, the effect of improving the accuracy of target recognition and the overall performance of the system is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of optoelectronic pod image processing, and particularly to an optoelectronic pod image enhancement system and method based on super-resolution. Background Art

[0002] As an important reconnaissance and target locking device, optoelectronic pods are widely used in unmanned aerial vehicles and aviation platforms. These pods are usually equipped with infrared sensors for detecting and identifying targets under various conditions. However, the native resolution of infrared sensors is usually low, which is particularly obvious under long-distance or low-light conditions, resulting in loss of image details and affecting the accuracy of target recognition.

[0003] Traditional super-resolution algorithms are mostly optimized for visible light images and do not fully consider the special characteristics of infrared images, such as spectral characteristics and thermal radiation distribution, which results in poor performance when processing infrared images.

[0004] In summary, the current application of super-resolution technology in improving the quality of pod infrared images still has obvious deficiencies. Therefore, it is very necessary to propose an optoelectronic pod image enhancement system and method that can improve the accuracy of target recognition and the overall performance of the system. Summary of the Invention

[0005] The purpose of the present invention is to provide an optoelectronic pod image enhancement system and method based on super-resolution, aiming to improve the accuracy of target recognition and the overall performance of the system.

[0006] To achieve the above purpose, an optoelectronic pod image enhancement method based on super-resolution adopted by the present invention includes the following steps:

[0007] Capture low-resolution infrared image data and output the low-resolution infrared image data;

[0008] Receive the low-resolution infrared image data and perform image quality optimization processing on the infrared image data;

[0009] Perform super-resolution processing on the image to convert the low-resolution infrared image data into high-resolution infrared image data;

[0010] Visualize the converted infrared image data.

[0011] Among them, in the step of performing super-resolution processing on the image to convert the low-resolution infrared image data into high-resolution infrared image data:

[0012] Set the network structure and train the super-resolution model;

[0013] Synthesize training data;

[0014] Analyze the low-resolution infrared image data according to the trained super-resolution model to generate a high-resolution image.

[0015] Among them, in the step of setting the network structure and training the super-resolution model:

[0016] The network structure includes a generator and a discriminator. The generator is used to convert the low-resolution image into a high-resolution image, and the discriminator is used to distinguish the generated image from the real image; the generator contains multiple residual nested dense blocks.

[0017] Among them, in the step of setting the network structure and training the super-resolution model, the process of training the super-resolution model is as follows:

[0018] Train the generator for peak signal-to-noise ratio using the L1 loss function;

[0019] Use a combination of L1 loss, perceptual loss, and GAN loss to train the generator and discriminator simultaneously.

[0020] Among them, in the step of synthesizing training data:

[0021] Adopt a second-order degradation model to generate training pairs for complex degradation simulation. The degradation process includes blurring, noise, resizing, and JPEG compression operations, and apply a sinc filter with a probability of 0.1 to synthesize ringing and overshoot artifacts.

[0022] The present invention also provides an optoelectronic pod image enhancement system based on super-resolution, including a sampling module, an optimization module, an infrared super-resolution module, and a display module, where:

[0023] The sampling module is used to capture low-resolution infrared image data and output the low-resolution infrared image data;

[0024] The optimization module is used to receive the low-resolution infrared image data and perform image quality optimization processing on the infrared image data;

[0025] The infrared super-resolution module is used to perform super-resolution processing on the image and convert the low-resolution infrared image data into high-resolution infrared image data;

[0026] The display module is used to visually display the converted infrared image data.

[0027] An image enhancement system and method for an optoelectronic pod based on super - resolution of the present invention respectively use the sampling module, the optimization module, the infrared super - resolution module and the display module to perform the following steps: capture low - resolution infrared image data and output the low - resolution infrared image data; receive the low - resolution infrared image data and perform image quality optimization processing on the infrared image data; perform super - resolution processing on the image to convert the low - resolution infrared image data into high - resolution infrared image data; perform visual display on the converted infrared image data; in the above manner, the effect of improving the accuracy of target recognition and the overall performance of the system is obtained. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following - described drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0029] Figure 1 is the flowchart of the steps of the method for enhancing the image of an optoelectronic pod based on super - resolution of the present invention.

[0030] Figure 2 is the flowchart of the steps of S300 of the present invention.

[0031] Figure 3 is the structural schematic diagram of the image enhancement system for an optoelectronic pod based on super - resolution of the present invention.

[0032] Figure 4 is the structural schematic diagram of the electronic device of the present invention.

[0033] 501 - Sampling module, 502 - Optimization module, 503 - Infrared super - resolution module, 504 - Display module. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0034] Here, the exemplary embodiments will be described in detail, and the examples are shown in the drawings. When the following description involves the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application.

[0035] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms "a", "the", and "said" used in this application and the appended claims are also intended to include the plural forms unless the context clearly dictates otherwise. It should also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0036] It should be understood that although the terms first, second, third, etc. may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "when" or "while" or "in response to determining".

[0037] Please refer to Figure 1 and Figure 2 , the present invention provides a method for enhancing the image of an optoelectronic pod based on super-resolution, including the following steps:

[0038] S100: Capture low-resolution infrared image data and output the low-resolution infrared image data;

[0039] S200: Receive the low-resolution infrared image data and perform image quality optimization processing on the infrared image data;

[0040] S300: Perform super-resolution processing on the image to convert the low-resolution infrared image data into high-resolution infrared image data;

[0041] S400: Perform visual display on the converted infrared image data.

[0042] In this embodiment, first, capture low-resolution infrared image data and output the low-resolution infrared image data; then receive the low-resolution infrared image data and perform image quality optimization processing on the infrared image data; then perform super-resolution processing on the image to convert the low-resolution infrared image data into high-resolution infrared image data; finally, perform visual display on the converted infrared image data; in this way, the effect of improving the accuracy of target recognition and the overall performance of the system is obtained.

[0043] Further, in the step of performing super-resolution processing on the image to convert the low-resolution infrared image data into high-resolution infrared image data:

[0044] S301: Set the network structure and perform training of the super-resolution model;

[0045] In this embodiment, the network structure includes a generator and a discriminator. The generator is used to convert a low-resolution image into a high-resolution image; the discriminator is used to distinguish between the generated image and the real image, and uses a UNet design with spectral normalization. The UNet structure can output authenticity values for each pixel through skip connections, provide detailed per-pixel feedback for the generator, help improve the discrimination ability for complex training outputs, and generate accurate gradient feedback to enhance local textures. Spectral normalization is used to stabilize the training dynamics and alleviate the over-sharpness and artifact problems brought by GAN training; among them, the generator contains multiple residual nested dense blocks.

[0046] The generator is trained using the L1 loss function for peak signal-to-noise ratio. By training on the DIV2K, Flickr2K, OutdoorSceneTraining, and the in-house infrared dataset, a total of 1000K iterations are performed, and exponential moving average (EMA) is used to improve training stability and performance.

[0047] A combination of L1 loss, perceptual loss, and GAN loss is used to train the generator and discriminator simultaneously; among them, the GAN loss is calculated based on the discriminator's discrimination results for the generated image and the real image, prompting the generator to generate high-quality images close to the real ones; the perceptual loss is calculated using the feature maps of {conv1,...conv5} in the pre-trained network (weights are {0.1, 0.1, 1, 1, 1}) to guide the generator to generate more natural images perceptually; the L1 loss helps to maintain the low-frequency information and overall structure of the image; a total of 400K iterations are performed, and EMA is also used.

[0048] S302: Synthesize training data;

[0049] In this embodiment, a second-order degradation model is used to generate training pairs for complex degradation simulation. The degradation process includes operations such as blurring, noise addition, resizing, and JPEG compression, and a sinc filter is applied with a probability of 0.1 to synthesize ringing and overshoot artifacts; among them, blurring uses multiple kernels such as Gaussian, generalized Gaussian, and plateau with probabilities of {0.7, 0.15, 0.15} respectively, the kernel size is randomly selected from {7, 9,...21}, and parameters such as the standard deviation are sampled within a certain range); the noise is Gaussian noise and Poisson noise with a probability of 0.5 each, and the parameter settings such as the noise intensity are within a certain range; resizing is randomly selected from area, bilinear, and bicubic algorithms; JPEG compression has a quality factor in [30, 95]; and a sinc filter is applied with a probability of 0.1 to synthesize ringing and overshoot artifacts, and some operations are skipped with a certain probability to increase diversity.

[0050] During the entire training process, the generator continuously attempts to generate high-quality images that can deceive the discriminator, while the discriminator continuously improves its discrimination ability, and the two play against each other. The generator adjusts its own parameters according to the feedback of the discriminator and the gradient of the loss function, and gradually learns how to better restore image details, reduce artifacts and noise, thereby improving the image quality.

[0051] S303: Analyze the low-resolution infrared image data according to the trained super-resolution model to generate a high-resolution image.

[0052] In this embodiment, analyzing the low-resolution infrared image data according to the trained super-resolution model to generate a high-resolution image can effectively suppress the noise in the infrared image and improve the clarity and recognizability of the image.

[0053] Corresponding to the foregoing embodiments of the super-resolution-based electro-optical pod image enhancement method, the present application also provides an embodiment of a super-resolution-based electro-optical pod image enhancement system.

[0054] Figure 3 is a block diagram of a super-resolution-based electro-optical pod image enhancement system shown according to an exemplary embodiment. Refer to Figure 3 , the system may include: a sampling module 501, an optimization module 502, an infrared super-resolution module 503, and a display module 504, where:

[0055] The sampling module 501 is configured to capture low-resolution infrared image data and output the low-resolution infrared image data;

[0056] The optimization module 502 is configured to receive the low-resolution infrared image data and perform image quality optimization processing on the infrared image data;

[0057] The infrared super-resolution module 503 is configured to perform super-resolution processing on the image and convert the low-resolution infrared image data into high-resolution infrared image data;

[0058] The display module 504 is configured to visually display the converted infrared image data.

[0059] In this embodiment, the sampling module 501 captures low-resolution infrared image data and outputs the low-resolution infrared image data; the optimization module 502 receives the low-resolution infrared image data and performs image quality optimization processing on the infrared image data; the infrared super-resolution module 503 performs super-resolution processing on the image and converts the low-resolution infrared image data into high-resolution infrared image data; the display module 504 visually displays the converted infrared image data; and the effect of improving the accuracy of target recognition and the overall performance of the system is obtained.

[0060] Regarding the system in the above embodiments, the specific manner in which each module performs operations has been described in detail in the embodiments related to the method, and will not be elaborated here.

[0061] For the system embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to the descriptions in the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this application. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0062] Correspondingly, this application also provides an electronic device, including: one or more processors; a memory for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the super-resolution-based optoelectronic pod image enhancement method as described above. As Figure 4 shown, it is a hardware structure diagram of a device with any data processing ability where the super-resolution-based optoelectronic pod image enhancement system provided by the embodiment of the present invention is located. In addition to Figure 4 the processors, memory, and network interfaces shown, any device with data processing ability where the device in the embodiment is located usually also includes other hardware according to the actual functions of the device with any data processing ability, which will not be elaborated here.

[0063] Correspondingly, this application also provides a computer-readable storage medium, on which computer instructions are stored. When the instructions are executed by a processor, the super-resolution-based optoelectronic pod image enhancement method as described above is implemented. The computer-readable storage medium can be an internal storage unit of any device with data processing ability described in any of the foregoing embodiments, such as a hard disk or memory. The computer-readable storage medium can also be an external storage device, such as a plug-in hard disk, a Smart Media Card (SMC), an SD card, a Flash Card, etc. equipped on the device. Further, the computer-readable storage medium can also include both the internal storage unit of any device with data processing ability and the external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by any device with data processing ability, and can also be used to temporarily store the data that has been output or will be output.

[0064] Other embodiments of the present application will be readily contemplated by those skilled in the art upon consideration of the specification and practice of the disclosure herein. The present application is intended to cover any variations, uses, or adaptations of the present application, which follow the general principles of the present application and include known common general knowledge or conventional technical means in the technical field not disclosed in the present application.

[0065] It should be understood that the present application is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope.

Claims

1. A method for photoelectric pod image enhancement based on super-resolution, characterized in that: The steps include: Capturing low-resolution infrared image data and outputting low-resolution infrared image data; Receive low-resolution infrared image data and optimize the image quality of the infrared image data; Performing super-resolution processing on the image to convert low-resolution infrared image data into high-resolution infrared image data; Visualize the converted infrared image data.

2. The method for photoelectric pod image enhancement based on super-resolution according to claim 1, characterized in that: In the step of performing super-resolution processing on an image and converting low-resolution infrared image data into high-resolution infrared image data: Set up the network structure and train the super-resolution model; Synthetic training data; The low-resolution infrared image data is analyzed according to the trained super-resolution model to generate a high-resolution image.

3. The method for photoelectric pod image enhancement based on super-resolution according to claim 2, characterized in that: In the steps of setting up the network structure and training the super-resolution model: The network structure includes a generator and a discriminator. The generator is used to convert low-resolution images into high-resolution images, and the discriminator is used to distinguish the generated images from the real images. The generator consists of multiple residual nested dense blocks.

4. The method for photoelectric pod image enhancement based on super-resolution according to claim 3, characterized in that: In the steps of setting up the network structure and training the super-resolution model, the process of super-resolution model training is: Train the generator using L1 loss function for peak signal-to-noise ratio; A combination of L1 loss, perceptual loss, and GAN loss is used to train the generator and discriminator simultaneously.

5. The method for photoelectric pod image enhancement based on super-resolution according to claim 2, characterized in that: In the step of synthesizing training data: A second-order degradation model is used to generate training pairs. Complex degradation simulation is performed. The degradation process includes blurring, noise, resizing and JPEG compression operations. A sinc filter is applied with a probability of 0.1 to synthesize ringing and overshoot artifacts.

6. An optoelectronic pod image enhancement system based on super-resolution, applied to the optoelectronic pod image enhancement method based on super-resolution as claimed in claim 1, characterized in that: It includes sampling module, optimization module, infrared super-resolution module and display module, among which: The sampling module is used to capture low-resolution infrared image data and output low-resolution infrared image data; The optimization module is used to receive low-resolution infrared image data and perform image quality optimization processing on the infrared image data; The infrared super-resolution module is used to perform super-resolution processing on the image, and convert low-resolution infrared image data into high-resolution infrared image data; The display module is used to visually display the converted infrared image data.

Images