Infrared image super-resolution reconstruction method and device, equipment and storage medium

Abstract

The invention relates to an infrared image super-resolution reconstruction method, device and equipment and a storage medium, and is applied to the technical field of image processing, and the method comprises the steps: obtaining an original infrared image which is an infrared image needing super-resolution reconstruction; performing resolution reduction processing on the original infrared image to generate a fuzzy infrared image; performing saliency analysis on the fuzzy infrared image by adopting an image saliency detection algorithm, and generating a saliency region image block and a non-saliency region image block; inputting the salient region image block into a trained sparse dictionary to generate a salient region super-division image; and splicing and fusing the salient region super-resolution image and the non-salient region image block to generate an infrared super-resolution reconstructed image. The method has the technical effects that the complexity of a super-resolution reconstruction algorithm of the infrared image is reduced, the calculation cost is reduced, and the definition of the image is improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method, device, equipment and storage medium for super-resolution reconstruction of infrared images. Background Art

[0002] Compared with white light images, infrared images can detect targets in adverse environments such as smoke, poor lighting, and night. However, the overall visual effect of infrared images is poor and the grayscale levels are unclear, which affects subsequent target recognition and detection. Improving the quality of infrared images through algorithms is a feasible way.

[0003] Super-resolution image reconstruction technology reconstructs high-resolution images through single-frame or multiple-frame low-resolution images; this technology has great research value in eliminating image quality degradation caused by motion blur, non-ideal sampling and imaging focusing.

[0004] In the current scheme, the algorithm for super-resolution reconstruction of infrared images is highly complex, resulting in overfitting and low efficiency of infrared image reconstruction. Summary of the invention

[0005] In order to help solve the problem of high complexity of infrared image super-resolution reconstruction algorithm, resulting in overfitting and low efficiency of infrared image reconstruction, the present application provides an infrared image super-resolution reconstruction method, device, equipment and storage medium.

[0006] In a first aspect, the present application provides an infrared image super-resolution reconstruction method, which adopts the following technical solution: the method comprises:

[0007] Acquire an original infrared image, where the original infrared image is an infrared image that needs to be reconstructed with super resolution;

[0008] Performing resolution reduction processing on the original infrared image to generate a blurred infrared image;

[0009] Using an image saliency detection algorithm to perform saliency analysis on the blurred infrared image, and generating saliency region image blocks and non-saliency region image blocks;

[0010] Inputting the salient region image block into the trained sparse dictionary to generate a salient region super-resolution image;

[0011] The super-resolution image of the salient region and the image block of the non-salient region are spliced ​​and fused to generate an infrared super-resolution reconstructed image.

[0012] In a specific implementation scheme, the lowering of the resolution of the original infrared image to generate a blurred infrared image includes:

[0013] Downsampling the original infrared image to generate a low-resolution infrared image;

[0014] The low-resolution infrared image is interpolated and amplified to generate the blurred infrared image.

[0015] In a specific implementation scheme, the adopting an image saliency detection algorithm to perform saliency analysis on the blurred infrared image and generate saliency region image blocks and non-saliency region image blocks comprises:

[0016] Using first-order and second-order gradient operators to extract features from the blurred infrared image to generate infrared image features;

[0017] An image saliency detection algorithm is used to distinguish the saliency of the infrared image features, and generate saliency region image blocks and non-saliency region image blocks.

[0018] In a specific implementation scheme, the step of using an image saliency detection algorithm to distinguish the infrared image features by saliency to generate salient area image blocks and non-salient area image blocks comprises:

[0019] Using an image saliency algorithm to calculate the saliency of pixels in the infrared image features;

[0020] Marking pixels in the infrared image features whose pixel saliency is greater than a preset value and generating salient points;

[0021] Calculating the proportion of the salient points in the infrared image features, and marking the area where the proportion of the salient points is greater than a preset proportion value as a salient area image block;

[0022] The infrared image features outside the salient region image block are marked as non-salient region image blocks.

[0023] In a specific implementation scheme, the training method of the sparse dictionary includes:

[0024] Performing sparse representation on the original infrared image and constructing a sparse representation optimization problem;

[0025] According to the sparse representation optimization problem, the K-SVD algorithm is used to perform dictionary training on the salient region image block, and the trained sparse dictionary is generated.

[0026] In a specific possible implementation scheme, the step of splicing and fusing the salient region super-resolution image and the non-salient region image block to generate an infrared super-resolution reconstructed image comprises:

[0027] Identify the overlapping area after the super-resolution image of the salient area and the image block of the non-salient area are spliced ​​and fused;

[0028] In the overlapping area, the regional maximum energy method is used to perform matching and fusion to generate the infrared super-resolution reconstructed image.

[0029] In a specific implementation manner, before the super-resolution image of the salient region and the image block of the non-salient region are spliced ​​and fused to generate the infrared super-resolution reconstructed image, the method further includes:

[0030] Performing gamma correction on the non-significant area image block to generate a non-significant area corrected image block;

[0031] The step of splicing and fusing the salient region super-resolution image and the non-salient region image block to generate an infrared super-resolution reconstructed image comprises:

[0032] The super-resolution image of the salient region and the corrected image block of the non-salient region are spliced ​​and fused to generate an infrared super-resolution reconstructed image.

[0033] In a second aspect, the present application provides an infrared image super-resolution reconstruction device, which adopts the following technical solution: the device comprises:

[0034] An original image acquisition module is used to acquire an original infrared image, where the original infrared image is an infrared image that needs to be reconstructed with super resolution;

[0035] A blurred image generation module, used for performing resolution reduction processing on the original infrared image to generate a blurred infrared image;

[0036] A saliency distinction module, used for performing saliency analysis on the blurred infrared image using an image saliency detection algorithm, and generating saliency region image blocks and non-saliency region image blocks;

[0037] A super-resolution image generation module, used for inputting the salient region image block into the trained sparse dictionary to generate a super-resolution image of the salient region;

[0038] The image fusion and reconstruction module is used to splice and fuse the super-resolution image of the salient area and the image block of the non-salient area to generate an infrared super-resolution reconstructed image.

[0039] In a third aspect, the present application provides a computer device, which adopts the following technical solution: it includes a memory and a processor, and the memory stores a computer program that can be loaded by the processor and execute any of the above-mentioned infrared image super-resolution reconstruction methods.

[0040] In a fourth aspect, the present application provides a computer-readable storage medium, which adopts the following technical solution: storing a computer program that can be loaded by a processor and execute any of the above-mentioned infrared image super-resolution reconstruction methods.

[0041] In summary, this application has the following beneficial technical effects:

[0042] This application scheme adopts a saliency reconstruction algorithm, which can improve the clarity of the target and suppress the background noise unique to infrared images; in addition, based on the concept of sparse coding, the algorithm computational complexity is reduced. This application scheme can solve problems such as partial detail loss, poor noise reduction effect, model complexity, and poor data versatility. The saliency super-resolution reconstruction strategy has the characteristics of lightweight. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 is a flow chart of the infrared image super-resolution reconstruction method in an embodiment of the present application;

[0044] Figure 2 is a schematic diagram of a sparse dictionary trained in an embodiment of the present application;

[0045] Figure 3 This is a complete flow chart of the infrared image super-resolution reconstruction process in the embodiment of the present application;

[0046] Figure 4 is a schematic diagram of an infrared image super-resolution reconstruction device in an embodiment of the present application;

[0047] Figure 5 It is a schematic diagram used to embody a computer device in an embodiment of the present application.

[0048] Reference numerals: 401, original image acquisition module; 402, blurred image generation module; 403, saliency distinction module; 404, super-resolution image generation module; 405, image fusion and reconstruction module. DETAILED DESCRIPTION

[0049] The following combination Figure 1-Figure 5 This application is described in further detail.

[0050] The embodiment of the present application discloses a super-resolution reconstruction method for infrared images, which is applied to an infrared image super-resolution reconstruction system. The method can realize super-resolution reconstruction of infrared images to improve the quality of infrared images. Compared with white light images, infrared images can detect targets in adverse environments such as smoke, poor lighting, and night. However, the overall visual effect of infrared images is not good and the grayscale levels are not clear, which may affect subsequent target recognition and detection. Therefore, it is necessary to process infrared images to improve the quality of infrared images, which is helpful for subsequent target recognition and detection.

[0051] In existing solutions, super-resolution reconstruction technology is generally used to reconstruct infrared images. Super-resolution image reconstruction technology reconstructs high-resolution images through single-frame or multi-frame low-resolution images; this technology has great research value in eliminating image quality degradation caused by motion blur, non-ideal sampling and imaging focus. However, in the existing solutions, the algorithm complexity of super-resolution reconstruction of infrared images is relatively high, which leads to overfitting and low efficiency of infrared image reconstruction. In order to help improve the efficiency of image reconstruction and avoid overfitting as much as possible, the present application provides a super-resolution reconstruction method for infrared images.

[0052] Reference Figure 1 , the method comprises the following steps:

[0053] S10, obtaining an original infrared image, where the original infrared image is an infrared image that needs to be reconstructed with super resolution.

[0054] Specifically, the system acquires the original infrared image, which is generally a high-resolution image. Since the overall visual effect of the original infrared image is not good and the grayscale levels are not clear, it has a certain impact on the subsequent target recognition and detection. Therefore, the infrared image can be reconstructed through super-resolution reconstruction to improve the quality of the infrared image.

[0055] S20, performing resolution reduction processing on the original infrared image to generate a blurred infrared image.

[0056] Specifically, the original image is a high-resolution image, and the original image is blurred and reduced in resolution to obtain a low-resolution blurred infrared image, which is then processed and reconstructed to generate the reconstructed image required by the user. Specifically, the original infrared image is downsampled to generate a low-resolution infrared image, and then the low-resolution infrared image is interpolated and amplified to generate the final blurred infrared image.

[0057] S30, performing saliency analysis on the blurred infrared image using an image saliency detection algorithm, and generating saliency region image blocks and non-saliency region image blocks.

[0058] Specifically, the obtained blurred infrared image is analyzed for saliency. Based on the classification dictionary method, the global contrast image saliency detection algorithm (LC) is used to calculate the global contrast of a certain pixel in the entire image entropy, that is, the sum of the grayscale distances between the pixel and other pixels as the saliency value of the pixel. Through the processing of the image saliency detection algorithm, the saliency area image blocks and the non-saliency area image blocks are generated.

[0059] S40, inputting the salient region image block into the trained sparse dictionary to generate a salient region super-resolution image.

[0060] Specifically, the sparse dictionary is processed on the image blocks of the salient regions, the sparse coding is solved, and the final salient region super-resolution image is iterated through the gradient descent method. Sparse dictionary is a technology widely used in signal processing, image processing, machine learning and other fields. It combines the advantages of base dictionary and learning dictionary, and can improve the structure and adaptability of the dictionary, thereby improving the efficiency and expression ability of the dictionary. The method of constructing sparse dictionary can improve the accuracy and computational efficiency of the algorithm.

[0061] S50, splicing and fusing the salient region super-resolution image and the non-salient region image blocks to generate an infrared super-resolution reconstructed image.

[0062] Specifically, after sparse dictionary processing is performed on the image blocks in the salient area, the obtained super-resolved image of the salient area and the image blocks in the non-salient area are spliced ​​and fused to generate the final infrared super-resolution reconstructed image.

[0063] In this application, considering that the super-resolution reconstruction effect of infrared images is not good and the reconstruction process is inevitably affected by high-frequency noise, a hierarchical analysis method is introduced, and a saliency analysis algorithm is used to distinguish between salient area images and non-salient images, which can reduce the problem of increased calculation amount, improve the clarity of the target, and suppress the background noise unique to infrared images. In addition, the use of sparse dictionaries for processing can reduce the complexity of algorithm calculations and improve the efficiency of super-resolution reconstruction of infrared images.

[0064] In one embodiment, the method of using an image saliency detection algorithm to perform saliency analysis on a blurred infrared image and generating saliency region image blocks and non-saliency region image blocks may be specifically performed as follows:

[0065] The first-order and second-order gradient operators are used to extract the features of the blurred infrared image and generate the infrared image features. Considering that the super-resolution reconstruction effect of the infrared image is not good, the reconstruction process is difficult to avoid the influence of high noise. The main feature areas of interest, such as the salient features of the infrared image, are extracted from the infrared image after resolution reduction, and the salient areas and non-salient areas are distinguished and processed separately, which can reduce the amount of calculation. At the same time, the salient reconstruction algorithm can improve the clarity of the target and suppress the background noise unique to the infrared image. Based on the above considerations, the image saliency detection algorithm is used to distinguish the saliency of the infrared image features and generate salient area image blocks and non-salient area image blocks; specifically, the global contrast image saliency detection algorithm (LC) is used to calculate the saliency of the pixels in the infrared image features, and the global contrast of a certain pixel in the entire image entropy is calculated, that is, the sum of the grayscale distances between the pixel and other pixels is taken as the saliency value of the pixel. For example, for a certain pixel I in an image Ik The calculation method of the significant value can be expressed as:

[0066] LC(I k )=Σ||I k -I||

[0067] LC(I k ) represents pixel I k The saliency value of image I.

[0068] After calculating the saliency value of each pixel in the infrared image feature extracted from the fuzzy infrared image, the pixels in the infrared image feature whose saliency is greater than the preset value are marked, and a salient point is generated. Then, the proportion of the salient points in the infrared image feature is calculated, and the area where the proportion of salient points is greater than the preset proportion value is marked as a salient area image block. Finally, the infrared image features outside the salient area image block are marked as a non-salient area image block. For example, in the embodiment of the present application, pixels with a saliency value greater than 0.7 are specifically marked as salient points, image blocks containing more than 40% salient points are marked as salient area image blocks, and those outside the salient area image blocks are marked as non-salient area image blocks. Among them, in the embodiment of the present application, only 0.7 and 40% are used as illustrations. In actual applications, the preset values ​​for marking salient points and the preset proportion values ​​for judging salient area image blocks can be set by the user according to actual conditions, and are not limited here.

[0069] In the present application scheme, the main feature area of ​​interest is extracted from the infrared image after resolution reduction, and the significant area and the non-significant area are distinguished in the image. The two areas are processed separately, which can reduce the amount of calculation. In addition, the use of a significant reconstruction algorithm can improve the clarity of the target and suppress the background noise unique to infrared images.

[0070] In one embodiment, the training method of the sparse dictionary may be specifically performed as follows:

[0071] The original infrared image is sparsely represented and the sparse representation optimization problem is constructed; specifically, the constraint model of super-resolution reconstruction can be expressed as:

[0072] Y=SHX

[0073] Wherein, Y represents a low-resolution image, H represents a blur filter, S represents a subsampling operator, and X represents a high-resolution image. In the embodiment of the present application, X is the original infrared image obtained. For the low-resolution image Y, there are many types of high-resolution images that can satisfy the constraint conditions of the above formula, thereby introducing a linear combination of sparsely represented image blocks to solve the constraint conditions.

[0074] For a high-resolution image block x, it can be represented as an image block dictionary DI The sparse linear combination of can be expressed as:

[0075] x≈D I α, α∈R k ,||α||<<k

[0076] Among them, x represents a high-resolution image block, D I It represents the image block dictionary, α represents the sparse coefficient, and k represents the norm of the sparse coefficient.

[0077] For a low-resolution image y, we can find the image patch dictionary D I The sparse representation of y is combined with the high-resolution image block, and the high-resolution image block is generated according to the sparse coefficients obtained by practical analysis. Corresponding to the optimal sparse representation of y, the constraint problem can be expressed as:

[0078]

[0079] Among them, G represents the linear feature extraction operator, η represents the set sparse threshold; the constraint problem of the optimal sparse representation of y is an indeterminate polynomial problem, which can be solved by minimizing the norm, and the constraint problem can be transformed into:

[0080]

[0081] Then it can be expressed by Lagrange polynomial transformation as follows:

[0082]

[0083] Among them, φ represents the balance factor between the coefficient sparsity of the sparse representation and the approximation to y, which has a great influence on the quality of the reconstructed image.

[0084] For a high-resolution image X, it can be represented by a subset of image blocks as follows:

[0085] X h ={X1,X2,...,X n}

[0086] The sparse representation optimization problem of the final constructed image can be expressed as:

[0087]

[0088] Among them, α is the sparse code, λ is the sparse weight, and D I is a sparse dictionary of the low-resolution image to be solved. By optimizing the sparse coding and sparse dictionary, a trained sparse dictionary is generated for the final reconstruction.

[0089] According to the constructed sparse representation optimization problem, the K-SVD algorithm is used to train the dictionary of the salient region image blocks, and a trained sparse dictionary is generated. Specifically, the salient region image blocks obtained after saliency distinction are clustered using K-means, and used as the basic dictionary for dictionary training. The image blocks after clustering are then input into the optimization equation, which can be expressed as:

[0090]

[0091] Among them, Z represents the clustered image block, α is the sparse coding, λ is the sparse weight, and D I is the sparse dictionary of the low-resolution image that needs to be solved.

[0092] After that, the optimization equation is solved by the orthogonal matching pursuit (OMP) algorithm to obtain the sparse code α, and the dictionary atoms are updated using singular value decomposition (SVD). After updating all the dictionary atoms, the sparse system is re-parsed, and the dictionary and coefficients are updated alternately multiple times. The iteration is repeated until convergence or the preset number of iterations is reached to construct the sparse dictionary of the image. The iteration method of the sparse coefficients can be expressed as:

[0093]

[0094] in, Indicates the sparse coefficients after training.

[0095] By combining the sparse coding α and the low-resolution dictionary D I The training is performed to obtain the final trained sparse dictionary and trained sparse coefficients. The high-resolution image can be reconstructed through the trained sparse dictionary and trained sparse coefficients. The reconstruction method can be expressed as:

[0096]

[0097] in, represents the sparse coefficient after training, Indicates the trained sparse dictionary. The trained sparse dictionary can be referenced Figure 2 shown.

[0098] In the present application, a sparse dictionary is trained and sparse representation is used to process image blocks in significant areas. The sparse representation method can reduce the complexity of algorithm calculations, reduce computational costs, and thus improve computational efficiency. In addition, the sparse representation method has strong robustness.

[0099] In one embodiment, the method of splicing and fusing the salient region super-resolution image and the non-salient region image blocks to generate the infrared super-resolution reconstructed image can be specifically performed as follows:

[0100] After the salient area image is processed to generate the salient area super-resolution image, the overlapping area after the salient area super-resolution image and the non-salient area image blocks are spliced ​​and fused is identified, and in the overlapping area, the regional maximum energy method is used to perform matching fusion to generate an infrared super-resolution reconstructed image; specifically, the energy of the overlapping area part is calculated, and the part with large energy in the overlapping area is selected as the source of the fused image.

[0101] In the present application scheme, the overlapping areas are selectively matched and fused using the regional maximum energy method to obtain the final super-resolution reconstructed image, which combines image fusion technology and image super-resolution reconstruction technology. Through this method, the complementary information between the significant super-resolution image and the non-significant area image blocks can be fully utilized to generate a high-quality infrared super-resolution reconstructed image.

[0102] In one embodiment, considering that the image blocks in the non-significant region also have noise, which may interfere with the infrared image reconstruction, the following steps may be performed before the super-resolution image in the salient region and the image blocks in the non-significant region are spliced ​​and fused to generate the infrared super-resolution reconstructed image:

[0103] Gamma correction is performed on the non-significant area image block to generate the non-significant area corrected image block; the super-resolution image of the salient area and the image block of the non-significant area are spliced ​​and fused to generate the infrared super-resolution reconstructed image, which includes: splicing and fusion of the super-resolution image of the salient area and the corrected image block of the non-significant area to generate the infrared super-resolution reconstructed image.

[0104] Gamma correction, also known as gamma nonlinearization or gamma encoding, is an algorithm for nonlinear tonal editing of images. The use of gamma correction algorithms can achieve better visual effects. For non-significant images, the gamma correction algorithm can be used to stretch the image grayscale and enhance the contrast image, thereby suppressing the interference of background noise on the background area.

[0105] Reference Figure 3, which is a complete flow chart of the infrared image super-resolution reconstruction process in the embodiment of the present application. First, the original infrared image is obtained and subjected to resolution reduction and blur processing operations to obtain a blurred infrared image, infrared features are extracted from the blurred infrared image, and significance analysis is performed on the extracted infrared feature image to determine whether it is a significant region image block. If it is a significant region image block, a sparse dictionary is trained on the significant region image block. After the sparse dictionary training is completed, the significant region image block is input into the trained sparse dictionary to generate a significant super-resolution image; if it is not a significant region image block, a gamma correction is performed on the non-significant region image block to obtain a corrected non-significant region image block, and the significant super-resolution image after sparse dictionary processing and the corrected non-significant region image block are spliced ​​and fused to obtain a final fused image, which is also the final infrared super-resolution reconstructed image.

[0106] Figure 1 FIG. 1 is a flow chart of a method for super-resolution reconstruction of infrared images in one embodiment. It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but the steps are not necessarily executed in the order indicated by the arrows; unless otherwise specified in this document, there is no strict order restriction for the execution of the steps, and the steps may be executed in other orders; and Figure 1 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0107] Based on the above method, an embodiment of the present application also discloses an infrared image super-resolution reconstruction device.

[0108] Reference Figure 4 , the device includes the following modules:

[0109] The original image acquisition module 401 is used to acquire an original infrared image, where the original infrared image is an infrared image that needs to be reconstructed with super resolution;

[0110] A blurred image generation module 402 is used to perform resolution reduction processing on the original infrared image to generate a blurred infrared image;

[0111] A saliency distinguishing module 403 is used to perform saliency analysis on the blurred infrared image using an image saliency detection algorithm, and generate saliency region image blocks and non-saliency region image blocks;

[0112] The super-resolution image generation module 404 is used to input the salient region image block into the trained sparse dictionary to generate a salient region super-resolution image;

[0113] The image fusion and reconstruction module 405 is used to merge and fuse the super-resolution image of the salient region and the image blocks of the non-salient region to generate an infrared super-resolution reconstructed image.

[0114] In one embodiment, the blurred image generation module 402 is specifically configured to downsample the original infrared image to generate a low-resolution infrared image; and perform interpolation and amplification processing on the low-resolution infrared image to generate a blurred infrared image.

[0115] In one embodiment, the super-resolution image generation module 404 is specifically used to use first-order and second-order gradient operators to extract features from blurred infrared images to generate infrared image features; use an image saliency detection algorithm to distinguish the saliency of infrared image features to generate salient area image blocks and non-salient area image blocks.

[0116] In one embodiment, the super-resolution image generation module 404 is specifically used to calculate the significance of pixels in infrared image features using an image significance algorithm; mark the pixels in the infrared image features whose significance is greater than a preset value and generate significant points; calculate the proportion of significant points in the infrared image features, and mark the area where the proportion of significant points is greater than a preset proportion value as a significant area image block; and mark the infrared image features outside the significant area image block as a non-significant area image block.

[0117] In one embodiment, the super-resolution image generation module 404 is also used to perform sparse representation on the original infrared image and construct a sparse representation optimization problem; according to the sparse representation optimization problem, the K-SVD algorithm is used to perform dictionary training on the salient area image blocks, and a trained sparse dictionary is generated.

[0118] In one embodiment, the image fusion and reconstruction module 405 is specifically used to identify the overlapping area after the salient area super-resolution image and the non-salient area image blocks are spliced ​​and fused; in the overlapping area, the regional maximum energy method is used to perform matching fusion to generate an infrared super-resolution reconstructed image.

[0119] In one embodiment, the image fusion and reconstruction module 405 is further used to perform gamma correction on the non-significant area image blocks to generate non-significant area corrected image blocks; splicing and fusing the super-resolution image of the significant area and the image blocks of the non-significant area to generate an infrared super-resolution reconstructed image includes: splicing and fusing the super-resolution image of the significant area and the corrected image blocks of the non-significant area to generate an infrared super-resolution reconstructed image.

[0120] The infrared image super-resolution reconstruction device provided in the embodiment of the present application can be applied to the infrared image super-resolution reconstruction method provided in the above embodiment. For relevant details, refer to the above method embodiment. The implementation principle and technical effect are similar and will not be repeated here.

[0121] It should be noted that: when the infrared image super-resolution reconstruction device provided in the embodiment of the present application performs infrared image super-resolution reconstruction, only the division of the above-mentioned functional modules / functional units is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules / functional units as needed, that is, the internal structure of the infrared image super-resolution reconstruction device is divided into different functional modules / functional units to complete all or part of the functions described above. In addition, the implementation method of the infrared image super-resolution reconstruction method provided in the above method embodiment and the implementation method of the infrared image super-resolution reconstruction device provided in this embodiment belong to the same concept. The specific implementation process of the infrared image super-resolution reconstruction device provided in this embodiment is detailed in the above method embodiment, which will not be repeated here.

[0122] The embodiment of the present application also discloses a computer device.

[0123] Specifically, if Figure 5 As shown, the computer device can be a computer device such as a desktop computer, a laptop computer, a PDA, and a cloud server. The computer device may include, but is not limited to, a processor and a memory. The processor and the memory may be connected via a bus or otherwise. The processor may be a central processing unit (CPU). The processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, graphics processors (GPU), embedded neural network processors (NPU) or other dedicated deep learning coprocessors, discrete gates or transistor logic devices, discrete hardware components and other chips, or a combination of the above-mentioned various chips.

[0124] As a non-transient computer-readable storage medium, the memory can be used to store non-transient software programs, non-transient computer executable programs and modules, such as program instructions / modules corresponding to the methods in the above-mentioned embodiments of the present application. The processor executes various functional applications and data processing of the processor by running the non-transient software programs, instructions and modules stored in the memory, that is, the method in the above-mentioned method implementation is realized. The memory may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created by the processor, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network and a combination thereof.

[0125] The embodiment of the present application also discloses a computer-readable storage medium.

[0126] Specifically, a computer-readable storage medium is used to store a computer program, and when the computer program is executed by a processor, the method in the above-mentioned method implementation is implemented. Those skilled in the art will understand that the implementation of all or part of the process in the above-mentioned implementation method of the present application can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and the program, when executed, may include the process of the implementation of the above-mentioned methods. Among them, the storage medium may be a disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory (Flash Memory), a hard disk (Hard Disk Drive, abbreviated as: HDD) or a solid-state drive (SSD), etc.; the storage medium may also include a combination of the above-mentioned types of memory.

[0127] This specific embodiment is merely an explanation of the present invention and is not a limitation of the present invention. After reading this specification, those skilled in the art may make non-creative modifications to the present embodiment as needed. However, as long as they are within the scope of the claims of the present invention, they are protected by the patent law.

Claims

1. A method for super-resolution reconstruction of infrared images, characterized in that: The method comprises: Acquire an original infrared image, where the original infrared image is an infrared image that needs to be reconstructed with super resolution; Performing resolution reduction processing on the original infrared image to generate a blurred infrared image; Using an image saliency detection algorithm to perform saliency analysis on the blurred infrared image, and generating saliency region image blocks and non-saliency region image blocks; Inputting the salient region image block into the trained sparse dictionary to generate a salient region super-resolution image; The super-resolution image of the salient region and the image block of the non-salient region are spliced ​​and fused to generate an infrared super-resolution reconstructed image.

2. The method according to claim 1, characterized in that: The step of reducing the resolution of the original infrared image to generate a blurred infrared image comprises: Downsampling the original infrared image to generate a low-resolution infrared image; The low-resolution infrared image is interpolated and amplified to generate the blurred infrared image.

3. The method according to claim 1, characterized in that: The adopting of an image saliency detection algorithm to perform saliency analysis on the blurred infrared image and generating saliency region image blocks and non-saliency region image blocks comprises: Using first-order and second-order gradient operators to extract features from the blurred infrared image to generate infrared image features; An image saliency detection algorithm is used to distinguish the saliency of the infrared image features, and generate saliency region image blocks and non-saliency region image blocks.

4. The method according to claim 3, characterized in that: The step of using an image saliency detection algorithm to distinguish the saliency of the infrared image features and generating saliency region image blocks and non-saliency region image blocks comprises: Using an image saliency algorithm to calculate the saliency of pixels in the infrared image features; Marking pixels in the infrared image features whose pixel saliency is greater than a preset value and generating salient points; Calculating the proportion of the salient points in the infrared image features, and marking the area where the proportion of the salient points is greater than a preset proportion value as a salient area image block; The infrared image features outside the salient region image block are marked as non-salient region image blocks.

5. The method according to claim 1, characterized in that: The training method of the sparse dictionary includes: Performing sparse representation on the original infrared image and constructing a sparse representation optimization problem; According to the sparse representation optimization problem, the K-SVD algorithm is used to perform dictionary training on the salient region image block, and the trained sparse dictionary is generated.

6. The method according to claim 1, characterized in that: The step of splicing and fusing the salient region super-resolution image and the non-salient region image block to generate an infrared super-resolution reconstructed image comprises: Identify the overlapping area after the super-resolution image of the salient area and the image block of the non-salient area are spliced ​​and fused; In the overlapping area, the regional maximum energy method is used to perform matching and fusion to generate the infrared super-resolution reconstructed image.

7. The method according to claim 1, characterized in that: Before the super-resolution image of the salient region and the image block of the non-salient region are spliced ​​and fused to generate the infrared super-resolution reconstructed image, the method further includes: Performing gamma correction on the non-significant area image block to generate a non-significant area corrected image block; The step of splicing and fusing the salient region super-resolution image and the non-salient region image block to generate an infrared super-resolution reconstructed image comprises: The super-resolution image of the salient region and the corrected image block of the non-salient region are spliced ​​and fused to generate an infrared super-resolution reconstructed image.

8. An infrared image super-resolution reconstruction device, characterized in that: The device comprises: An original image acquisition module (401) is used to acquire an original infrared image, wherein the original infrared image is an infrared image that needs to be reconstructed with super resolution; A blurred image generation module (402) is used to perform resolution reduction processing on the original infrared image to generate a blurred infrared image; A saliency distinguishing module (403) is used to perform saliency analysis on the blurred infrared image using an image saliency detection algorithm, and generate saliency region image blocks and non-saliency region image blocks; A super-resolution image generation module (404) is used to input the salient region image block into the trained sparse dictionary to generate a salient region super-resolution image; The image fusion and reconstruction module (405) is used to splice and fuse the super-resolution image of the salient region and the image block of the non-salient region to generate an infrared super-resolution reconstructed image.

9. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program which can be loaded by the processor and executes the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: A computer program is stored which can be loaded by a processor and execute the method according to any one of claims 1 to 7.

Images