Local discrimination super-resolution reconstruction method based on multi-order infrared image degradation simulation

Abstract

The invention provides a local discrimination super-resolution reconstruction method based on multi-order infrared image degradation simulation, and the method comprises the steps: 1, inputting an original infrared image into a multi-order infrared image degradation model, and obtaining a low-resolution degraded image; 2, inputting the low-resolution degraded image into a generator of a super-resolution reconstruction network to generate a super-resolution reconstruction image; 3, after feature fusion is carried out on the super-resolution reconstruction image and the original infrared image, the super-resolution reconstruction image and the original infrared image are input into a local discriminator of the super-resolution reconstruction network to be processed, and local discrimination loss is obtained; 4, performing weighted summation on the local discrimination loss and the preset model loss to obtain image reconstruction training loss; when the image reconstruction training loss is greater than the minimum loss value, returning to the step 2; and otherwise, outputting the trained super-resolution reconstruction network. According to the method, the deviation of local details can be carefully captured, the local details can be better optimized by a supervision generator, and the texture detail recovery accuracy in a complex scene is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of digital image processing, and particularly to a local discriminative super-resolution reconstruction method based on multi-order infrared image degradation simulation. Background Art

[0002] In the context of current technological development, infrared images have extensive applications in fields such as military, security, medical, and industrial. However, due to factors such as hardware limitations and cost considerations, the resolution of infrared imaging devices is often relatively low. This results in the loss and blurring of detailed information in infrared images, restricting their accuracy and reliability in tasks such as target recognition, target tracking, and anomaly detection. To address this issue, super-resolution reconstruction technology has become a popular research direction. However, existing deep learning-based super-resolution reconstruction methods, which construct deep neural networks to enable the model to learn the end-to-end mapping relationship from low-resolution infrared images to high-resolution infrared images and automatically extract image features to achieve super-resolution reconstruction, still have the following problems: The super-resolution reconstruction algorithm may introduce blurring or artifact effects, causing the image to lose its sense of reality. This is because the algorithm cannot accurately recover the missing detailed information or over-smooths the image. Therefore, the present invention proposes a local discriminative super-resolution reconstruction method based on multi-order infrared image degradation simulation. Summary of the Invention

[0003] The present invention provides a local discriminative super-resolution reconstruction method based on multi-order infrared image degradation simulation to solve the above problems.

[0004] The present invention provides a local discriminative super-resolution reconstruction method based on multi-order infrared image degradation simulation, including:

[0005] Step 1: Input the original infrared image into a multi-order infrared image degradation model to obtain a low-resolution degraded image;

[0006] Step 2: Input the low-resolution degraded image into the generator of the super-resolution reconstruction network to generate a super-resolution reconstructed image;

[0007] Step 3: After fusing the features of the super-resolution reconstructed image and the original infrared image, input them into the local discriminator of the super-resolution reconstruction network for processing, and obtain a local discriminative loss;

[0008] Step 4: Perform weighted summation on the local discriminative loss and a preset model loss to obtain an image reconstruction training loss for the super-resolution reconstructed image;

[0009] When the image reconstruction training loss is greater than the minimum loss value, return to Step 2;

[0010] Otherwise, end the training and output the trained super-resolution reconstruction network.

[0011] Preferably, in a local discriminative super-resolution reconstruction method based on multi-order infrared image degradation simulation, step 1 includes:

[0012] While inputting the original infrared image into the multi-order infrared image degradation model, the multi-order infrared image degradation model randomly arranges various image degradation strategies based on the image degradation strategy database to generate a multi-order infrared image degradation process;

[0013] Based on the multi-order infrared image degradation process, process the original infrared image to obtain a low-resolution degraded image;

[0014] Among them, the multi-order infrared image degradation model is a third-order degradation model.

[0015] Preferably, in a local discriminative super-resolution reconstruction method based on multi-order infrared image degradation simulation, step 2 includes:

[0016] After the low-resolution infrared image is input after the first convolution process, process it based on N RRDB modules to obtain a first output image;

[0017] After the second convolution process on the first output image, perform upsampling to obtain an upsampled image;

[0018] Based on the residual dense network, respectively obtain the image features extracted from each network layer of the upsampled image and the low-resolution infrared image to obtain the upsampled image features and the original image features of the low-resolution infrared image;

[0019] Perform convolution fusion on the upsampled image features and the original image features to obtain a super-resolution reconstructed image.

[0020] Preferably, in a local discriminative super-resolution reconstruction method based on multi-order infrared image degradation simulation, step 3 includes:

[0021] Perform non-overlapping sliding on the super-resolution reconstructed image based on a preset rectangular window, calculate the local entropy corresponding to the preset window, and determine whether the local entropy is greater than the local entropy threshold;

[0022] If so, based on the error value between the local entropy and the local entropy threshold, adaptively adjust the rectangular window, and perform comparison calculation on the pixel data within the adjusted rectangular window and the pixel data corresponding to the original infrared image to obtain a local residual map;

[0023] Otherwise, perform comparison calculation on the pixel data within the preset rectangular window and the pixel data corresponding to the original infrared image to obtain a local residual map;

[0024] Based on the local residual map, calculate the local variance of each pixel point within the adjusted rectangular window or the preset rectangular frame. When the local variance is greater than the preset variance, determine that the current pixel point is a pseudo-image pixel point;

[0025] Otherwise, determine that the current pixel point is a real-image pixel point;

[0026] Based on the local variances corresponding to all the pseudo-image pixel points in the super-resolution reconstruction image, calculate and obtain the local discriminant loss.

[0027] Preferably, in a local discriminant super-resolution reconstruction method based on multi-order infrared image degradation simulation, based on the error value between the local entropy and the local entropy threshold, adaptively adjust the rectangular window, including:

[0028] Based on big data technology, collect the window information entropy corresponding to infrared images of rectangular windows of different sizes;

[0029] Group the window information entropy based on the window size to obtain rectangular window data groups of multiple sizes, determine the window information entropy range corresponding to each rectangular window data group, and determine whether there is an overlapping range among the window information entropy ranges corresponding to rectangular windows of different sizes;

[0030] If not, use the window information entropy ranges corresponding to the current rectangular windows of different sizes as the final window information entropy ranges;

[0031] If so, obtain the rectangular window data groups corresponding to the rectangular windows with overlapping ranges as the first target data group and the second target data group respectively, perform clustering on the data within the first target data group and the second target data group respectively to obtain the first clustering center and the second clustering center;

[0032] Based on the first clustering center and the second clustering center, determine the target window information entropy value corresponding to the center point, and determine whether the target window information entropy value is within the overlapping range. If it is, based on the target window information entropy value, correct the upper limit value of the window information entropy range corresponding to the first target data group and the lower limit value of the window information entropy range corresponding to the second target data group respectively;

[0033] Otherwise, when the target window information entropy value is less than or equal to the lower limit value of the overlapping range, correct the lower limit value of the window information entropy range corresponding to the second target data group based on the lower limit value of the overlapping range;

[0034] When the target window information entropy value is greater than or equal to the upper limit value of the overlapping range, correct the lower limit value of the window information entropy range corresponding to the first target data group based on the upper limit value of the overlapping range;

[0035] Based on the correction results, determine the final window information entropy ranges corresponding to rectangular windows of different sizes;

[0036] Determine the window information entropy difference interval between rectangular windows of different sizes according to the range of the final window information entropy corresponding to rectangular windows of different sizes;

[0037] Match the error value between the local entropy and the local entropy threshold with the window information entropy difference intervals between multiple-sized rectangular windows and a preset rectangular window, determine the size corresponding to the optimal rectangular window, and adaptively adjust the window size of the preset rectangular window according to the size corresponding to the optimal rectangular window.

[0038] Preferably, in a local discriminant super-resolution reconstruction method based on multi-order infrared image degradation simulation, calculate the local discriminant loss based on the local variances corresponding to all pseudo image pixels in the super-resolution reconstruction image, including:

[0039] Calculate the standard local determination loss corresponding to the super-resolution reconstruction image based on a preset local discriminant loss model, mark the pseudo image pixels in the super-resolution reconstruction image, and extract the distribution characteristics of the pseudo image pixels based on the marking results;

[0040] Evaluate each image element in the super-resolution reconstruction image based on the distribution characteristics, and determine the image distortion rate of the super-resolution reconstruction image according to the evaluation results;

[0041] Evaluate the global reconstruction error of the super-resolution reconstruction image based on the image distortion rate. When the reconstruction error is greater than the preset error, obtain the global reconstruction error index based on the error rate between the reconstruction error and the preset error, and correct the standard local discriminant error based on the global reconstruction error index to obtain the output local discriminant error;

[0042] Otherwise, use the standard local discriminant error as the output local discriminant error.

[0043] Preferably, in a local discriminant super-resolution reconstruction method based on multi-order infrared image degradation simulation, evaluate each image element in the super-resolution reconstruction image based on the distribution characteristics, including:

[0044] Perform edge detection on the super-resolution reconstruction image, and divide the image information on the super-resolution reconstruction image into multiple image element information based on the edge detection results;

[0045] Based on the distribution characteristics of the pseudo image pixels and combined with the image element information segmentation results, determine the correlation relationship of the image representation information of each pseudo image on the effective image;

[0046] Based on the correlation relationship, determine the number of pseudo image pixels and the position distribution sub-characteristics on each image element;

[0047] Obtain the total number of pixels corresponding to each image element with pseudo-image pixels respectively, and obtain the artifact index of the corresponding image element according to the total number of pixels and the number of pseudo-image pixels of the corresponding image element;

[0048] When the artifact index is greater than or equal to a preset threshold, it is determined that the corresponding image element is distorted;

[0049] Otherwise, based on the position distribution sub-feature corresponding to the pseudo-image pixels on the corresponding image element, determine whether there is a concentrated area of pseudo-image pixels on the corresponding image element. If so, obtain the element distortion ratio based on the ratio of the number of pseudo-image pixels in the concentrated area of pseudo-image pixels to the total number of pixels corresponding to the corresponding image element;

[0050] Based on the element distortion ratio, determine the maximum concentrated area of pseudo-image pixels. When the element distortion ratio corresponding to the maximum concentrated area of pseudo-image pixels is greater than or equal to the threshold distortion ratio, it is determined that the corresponding image element is distorted;

[0051] Otherwise, it is determined that the corresponding image element is not distorted.

[0052] Preferably, in a local discriminative super-resolution reconstruction method based on multi-order infrared image degradation simulation, according to the evaluation result, determine the image distortion rate of the super-resolution reconstruction image, including:

[0053] Traverse the image elements on the super-resolution reconstruction image to determine the total number of image elements corresponding to the super-resolution reconstruction image and the number of distorted image elements;

[0054] Based on the total number of image elements corresponding to the super-resolution reconstruction image and the number of distorted image elements, obtain the distortion element ratio of the super-resolution reconstruction image, and use the distortion element ratio as the image distortion rate of the super-resolution reconstruction image.

[0055] Preferably, in a local discriminative super-resolution reconstruction method based on multi-order infrared image degradation simulation, it further includes:

[0056] The user inputs the low-resolution infrared image to be processed into the trained super-resolution reconstruction network for processing to obtain a super-resolution reconstruction infrared image;

[0057] And send the super-resolution reconstruction infrared image to the corresponding user terminal for display based on a preset sending address.

[0058] Preferably, in a local discriminative super-resolution reconstruction method based on multi-order infrared image degradation simulation, it further includes:

[0059] Based on big data, obtain a large number of high-resolution infrared images as the original infrared image to establish a training database, and collect the output super-resolution reconstruction images corresponding to the trained super-resolution reconstruction network to update the training database;

[0060] When the data update amount of the training database reaches the minimum update threshold, based on the latest training database, a plurality of original infrared images are randomly selected, and steps 1-4 are executed.

[0061] Other features and advantages of the present invention will be described in the following specification, and, in part, will be obvious from the specification, or will be understood by implementing the present invention. The objectives and other advantages of the present invention can be achieved and obtained by the structure specifically pointed out in the present application document.

[0062] The technical solution of the present invention will be further described in detail below with reference to the drawings and embodiments. Description of the Drawings

[0063] The drawings are used to provide a further understanding of the present invention, and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention, and do not constitute a limitation to the present invention. In the drawings:

[0064] Figure 1 is a flowchart of a local discriminant super-resolution reconstruction method based on multi-order infrared image degradation simulation of the present invention;

[0065] Figure 2 is a flowchart of step 1 of a local discriminant super-resolution reconstruction method based on multi-order infrared image degradation simulation of the present invention;

[0066] Figure 3 is a flowchart of step 2 of a local discriminant super-resolution reconstruction method based on multi-order infrared image degradation simulation of the present invention. Detailed Embodiments

[0067] The following describes the preferred embodiments of the present invention with reference to the drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0068] Embodiment 1:

[0069] The present invention provides a local discriminant super-resolution reconstruction method based on multi-order infrared image degradation simulation, as Figure 1 shown, including:

[0070] Step 1: Input the original infrared image into a multi-order infrared image degradation model to obtain a low-resolution degraded image;

[0071] Step 2: Input the low-resolution degraded image into the generator of the super-resolution reconstruction network to generate a super-resolution reconstructed image;

[0072] Step 3: After fusing the features of the super-resolution reconstructed image and the original infrared image, input them into the local discriminator of the super-resolution reconstruction network for processing, and obtain a local discriminant loss;

[0073] Step 4: Perform weighted summation on the local discrimination loss and the preset model loss to obtain the image reconstruction training loss of the super-resolution reconstructed image;

[0074] When the image reconstruction training loss is greater than the minimum loss value, return to Step 2;

[0075] Otherwise, end the training and output the trained super-resolution reconstruction network.

[0076] In this embodiment, the preset model loss is obtained by multiplying and summing the per-pixel reconstruction loss, the perceptual loss, and the adversarial loss with their corresponding balance parameters respectively. The balance parameters corresponding to the three are usually set to 0.01, 1, and 0.005 respectively.

[0077] Beneficial effects of the above technical solution: The present invention generates a low-resolution degraded image through a multi-stage infrared image degradation model, which can highly simulate various degradation situations that an infrared image may encounter during actual acquisition, transmission, etc., such as noise interference, downsampling blur, etc. The degradation simulation makes the subsequent training and reconstruction of the super-resolution reconstruction network more targeted. Compared with directly processing the original clear image, the reconstructed super-resolution image can better meet the requirements for improving image quality in the actual application scenario. Subsequently, the low-resolution degraded image is input into the generator of the super-resolution reconstruction network to generate a super-resolution reconstructed image. After fusing the features of the super-resolution reconstructed image and the original infrared image, it is input into the local discriminator of the super-resolution reconstruction network for processing to obtain the local discrimination loss of the super-resolution reconstructed image, and perform weighted summation on the local discrimination loss and the preset model loss to obtain the image reconstruction training loss of the super-resolution reconstructed image. Then, it is judged whether the super-resolution reconstruction network is trained through the image reconstruction loss, which can combine the enhanced details in the reconstructed image with the inherent feature information of the original image, enabling the subsequent discriminator to more accurately judge the local quality of the reconstructed image, and using the local discriminator to process the fused features, focusing on evaluating the authenticity and accuracy of the local area of the reconstructed image, being able to more carefully capture the deviation of local details. The addition of the local disk loss enriches the content of the overall model's training loss, can better supervise the generator to better optimize local details, improve the accuracy of texture detail restoration in complex scenarios, and enhance the overall visual quality of the reconstructed image, obtaining a super-resolution image that is closest to the real display content of the infrared image.

[0078] Embodiment 2:

[0079] Based on Embodiment 1, Step 1, as Figure 2 shown, includes:

[0080] Step 101: While inputting the original infrared image into the multi - order infrared image degradation model, the multi - order infrared image degradation model randomly arranges various image degradation strategies based on the image degradation strategy database to generate a multi - order infrared image degradation process;

[0081] Step 102: Process the original infrared image based on the multi - order infrared image degradation process to obtain a low - resolution degraded image;

[0082] Among them, the multi - order infrared image degradation model is a third - order degradation model.

[0083] In this embodiment, the image degradation strategy database includes but is not limited to noise interference (such as Gaussian noise, salt - and - pepper noise, Poisson noise, etc.), blur degradation (such as motion blur, defocus blur, Gaussian blur, etc.), geometric distortion (such as rotation / scaling distortion, perspective distortion, barrel / pincushion distortion, etc.), color distortion (such as white - balance error, abnormal color saturation, channel imbalance, etc.), compression artifacts (such as blocking effect, ringing effect, etc.), low - illumination degradation (such as vignetting, low signal - to - noise ratio, etc.).

[0084] Beneficial effects of the above - mentioned technical solution: The present invention generates a low - resolution degraded image through a multi - order infrared image degradation model, which can highly simulate various degradation situations that infrared images may encounter during actual acquisition, transmission, etc., such as noise interference, down - sampling blur, etc. The degradation simulation makes the training and reconstruction of the subsequent super - resolution reconstruction network more targeted. Compared with directly processing the original clear image, the reconstructed super - resolution image can better meet the requirements for image quality improvement in the actual application scenario. At the same time, it also overcomes the limitation of the training data volume of the super - resolution reconstruction network, providing a basis for a large amount of data training of the super - resolution reconstruction network.

[0085] Embodiment 3:

[0086] Based on Embodiment 1, Step 2, as Figure 3 shown, includes:

[0087] Step 201: After the low - resolution infrared image is input after the first convolution process, it is processed based on N RRDB modules to obtain a first output image;

[0088] Step 202: After the second convolution process on the first output image, perform up - sampling to obtain an up - sampled image;

[0089] Step 203: Based on the residual dense network, respectively obtain the image features extracted from each network layer of the up - sampled image and the low - resolution infrared image to obtain the up - sampled image features and the original image features of the low - resolution infrared image;

[0090] Step 204: Perform convolutional fusion on the upsampled image features and the original image features to obtain a super-resolution reconstructed image.

[0091] In this embodiment, inside the N RRDB modules, the output of each layer is connected to the input of subsequent layers through a dense connection method.

[0092] In this embodiment, the dense block (Dense block, i.e., the RRDB module) is composed of a convolutional layer and a LeakyRelu activation function. The convolutional layer is mainly used to extract feature maps, and the activation function is mainly to increase the non-linearity of the network. Thanks to the RRDB structure, the generator can capture more complex features and enhance the ability to restore image details.

[0093] In this embodiment, the convolutional kernels of the first convolutional process and the second convolutional process may be the same or different, and can be customized according to the actual needs of users.

[0094] Among them, the first convolutional process is a single convolution, and the second convolutional process is multiple consecutive convolutions. The convolutional kernels of the multiple consecutive convolutions may be the same or different.

[0095] Beneficial effects of the above technical solution: After the low-resolution infrared image is input after the first convolutional process in the present invention, it is processed based on N RRDB modules to obtain a first output image; after the second convolutional process on the first output image, upsampling is performed to obtain an upsampled image; based on the residual dense network, the image features extracted from each network layer of the upsampled image and the low-resolution infrared image are obtained, and the upsampled image features and the original image features of the low-resolution infrared image are obtained; convolutional fusion is performed on the upsampled image features and the original image features to obtain a super-resolution reconstructed image, realizing the autonomous reconstruction of the low-resolution infrared image.

[0096] Embodiment 4:

[0097] Based on Embodiment 1, Step 3 includes:

[0098] Perform non-overlapping sliding on the super-resolution reconstructed image based on a preset rectangular window, calculate the local entropy corresponding to the preset window, and determine whether the local entropy is greater than the local entropy threshold;

[0099] If so, based on the error value between the local entropy and the local entropy threshold, adaptively adjust the rectangular window, and calculate the comparison between the pixel data within the adjusted rectangular window and the pixel data corresponding to the original infrared image to obtain a local residual map;

[0100] Otherwise, calculate the comparison between the pixel data within the preset rectangular window and the pixel data corresponding to the original infrared image to obtain a local residual map;

[0101] Based on the local residual map, calculate the local variance of each pixel point within the adjusted rectangular window or the preset rectangular frame. When the local variance is greater than the preset variance, determine that the current pixel point is a pseudo-image pixel point;

[0102] Otherwise, determine that the current pixel point is a true-image pixel point;

[0103] Based on the local variances corresponding to all the pseudo-image pixel points in the super-resolution reconstructed image, calculate and obtain the local discriminant loss;

[0104] Among them, based on the error value between the local entropy and the local entropy threshold, adaptively adjust the rectangular window, including:

[0105] Based on big data technology, collect the window information entropy corresponding to infrared images of rectangular windows of different sizes;

[0106] Group the window information entropy based on the window size to obtain rectangular window data groups of multiple sizes, determine the window information entropy range corresponding to each rectangular window data group, and determine whether there is an overlapping range in the window information entropy ranges corresponding to rectangular windows of different sizes;

[0107] If not, use the window information entropy ranges corresponding to the current rectangular windows of different sizes as the final window information entropy ranges;

[0108] If so, obtain the rectangular window data groups corresponding to the rectangular windows with overlapping ranges as the first target data group and the second target data group respectively, perform clustering on the data within the first target data group and the second target data group respectively to obtain the first clustering center and the second clustering center;

[0109] Based on the first clustering center and the second clustering center, determine the target window information entropy value corresponding to the center point, and determine whether the target window information entropy value is within the overlapping range. If it is, based on the target window information entropy value, correct the upper limit value of the window information entropy range corresponding to the first target data group and the lower limit value of the window information entropy range corresponding to the second target data group respectively;

[0110] Otherwise, when the target window information entropy value is less than or equal to the lower limit value of the overlapping range, correct the lower limit value of the window information entropy range corresponding to the second target data group based on the lower limit value of the overlapping range;

[0111] When the target window information entropy value is greater than or equal to the upper limit value of the overlapping range, correct the lower limit value of the window information entropy range corresponding to the first target data group based on the upper limit value of the overlapping range;

[0112] Based on the correction results, determine the final window information entropy ranges corresponding to rectangular windows of different sizes;

[0113] Determine the window information entropy difference interval between rectangular windows of different sizes according to the range of the final window information entropy corresponding to rectangular windows of different sizes.

[0114] Match the error value between the local entropy and the local entropy threshold with the window information entropy difference intervals between multiple-sized rectangular windows and a preset rectangular window, determine the size corresponding to the optimal rectangular window, and adaptively adjust the window size of the preset rectangular window according to the size corresponding to the optimal rectangular window.

[0115] Beneficial effects of the above technical solution: Based on a preset rectangular window, perform non-overlapping sliding on the super-resolution reconstructed image, calculate the local entropy corresponding to the preset window, and determine whether the local entropy is greater than the local entropy threshold; if so, based on the error value between the local entropy and the local entropy threshold, adaptively adjust the rectangular window, while ensuring accurate capture of local features of the image, ensuring that the regional information in each rectangular frame reaches a certain standard, effectively improving the accuracy of subsequent local variance calculation, and then obtaining the pixel data in the rectangular window and comparing and calculating it with the pixel data corresponding to the original infrared image to obtain a local residual map; based on the local residual map, calculate the local variance of each pixel point in the adjusted rectangular window or the preset rectangular frame, and when the local variance is greater than the preset variance, determine that the current pixel point is a false image pixel point; otherwise, determine that the current pixel point is a true image pixel point, realizing the discrimination of false image pixel points in a local area, and calculating and obtaining a local discrimination loss based on the local variances corresponding to all false image pixel points in the super-resolution reconstructed image, providing a basis for further training of the super-resolution reconstruction network.

[0116] The present invention is based on

[0117] Example 5:

[0118] On the basis of Example 4, calculate and obtain a local discrimination loss based on the local variances corresponding to all false image pixel points in the super-resolution reconstructed image, including:

[0119] Calculate the standard local determination loss corresponding to the super-resolution reconstructed image based on a preset local discrimination loss model, mark the false image pixel points in the super-resolution reconstructed image, and based on the marking result, extract the distribution characteristics of the false image pixel points.

[0120] Based on the distribution characteristics, evaluate each image element in the super-resolution reconstructed image, and according to the evaluation result, determine the image distortion rate of the super-resolution reconstructed image.

[0121] Evaluate the global reconstruction error of the super-resolution reconstructed image based on the image distortion rate. When the reconstruction error is greater than the preset error, obtain the global reconstruction error index based on the error rate between the reconstruction error and the preset error, and correct the standard local discrimination error based on the global reconstruction error index to obtain the output local discrimination error;

[0122] Otherwise, use the standard local discrimination error as the output local discrimination error.

[0123] Among them, evaluating each image element in the super-resolution reconstructed image includes:

[0124] Perform edge detection on the super-resolution reconstructed image. Based on the edge detection results, divide the image information on the super-resolution reconstructed image into multiple image element information;

[0125] Based on the distribution characteristics of the pseudo-pixel points, combined with the image element information segmentation results, determine the correlation relationship of the image representation information of each pseudo-pixel point on the effective image;

[0126] Based on the correlation relationship, determine the number of pseudo-pixel points and the position distribution sub-characteristics on each image element;

[0127] Respectively obtain the total number of pixels corresponding to each image element with pseudo-pixel points. According to the total number of pixels and the number of pseudo-pixel points of the corresponding image element, obtain the artifact index of the corresponding image element;

[0128] When the artifact index is greater than or equal to the preset threshold, determine that the corresponding image element is distorted;

[0129] Otherwise, based on the position distribution sub-characteristics corresponding to the pseudo-pixel points on the corresponding image element, determine whether there is a concentrated area of pseudo-pixel points on the corresponding image element. If so, obtain the element distortion ratio based on the ratio of the number of pseudo-pixel points in the concentrated area of pseudo-pixel points to the total number of pixels corresponding to the corresponding image element;

[0130] Based on the element distortion ratio, determine the largest concentrated area of pseudo-pixel points. When the element distortion ratio corresponding to the largest concentrated area of pseudo-pixel points is greater than or equal to the threshold distortion ratio, determine that the corresponding image element is distorted;

[0131] Otherwise, determine that the corresponding image element is not distorted;

[0132] Among them, according to the evaluation results, determining the image distortion rate of the super-resolution reconstructed image includes:

[0133] Traverse the image elements on the super-resolution reconstructed image to determine the total number of image elements corresponding to the super-resolution reconstructed image and the number of distorted image elements;

[0134] Based on the total number of image elements corresponding to the super-resolution reconstructed image and the number of distorted image elements, obtain the distortion element ratio of the super-resolution reconstructed image, and use the distortion element ratio as the image distortion rate of the super-resolution reconstructed image.

[0135] Beneficial effects of the above technical solution: By marking the pseudo-pixel points in the super-resolution reconstructed image and extracting their distribution characteristics, the present invention can accurately locate and analyze the regions and patterns where artifacts appear in the image, which helps to deeply understand the distribution law of artifacts in the image and provides detailed and targeted information for subsequent evaluation. Then, based on the distribution characteristics of the pseudo-pixel points, each image element is evaluated, and then the image distortion rate is determined, which can comprehensively and objectively reflect the difference degree between the super-resolution reconstructed image and the original image. The evaluation method from the microscopic pixel level to the macroscopic image level can more accurately measure the quality loss of the image. Then, the reconstruction error of the super-resolution reconstructed image is evaluated using the image distortion rate, establishing a connection between the image quality and the reconstruction error, and the deviation degree from the ideal reconstruction result due to factors such as artifacts during the reconstruction process can be intuitively understood, providing a quantitative basis for judging the effectiveness of the reconstruction algorithm. When the reconstruction error is greater than the preset error, the standard local determination loss is corrected by combining the error rate between the reconstruction error and the preset error. This adaptive adjustment method can optimize the loss function of the model according to the error situation of the actual reconstruction result, enabling the model to pay more attention to the regions with artifacts and larger reconstruction errors during the subsequent training process, thereby reducing artifacts targeted and improving the quality of the reconstructed image. Moreover, by continuously localizing the discrimination loss, the adjustment of the loss during the entire training process is realized, enabling the model to gradually learn how to avoid generating artifacts and improve the accuracy of the reconstructed image, which helps the model to continuously optimize its own parameters during the training process, thereby improving the overall performance of the super-resolution reconstruction algorithm.

[0136] Embodiment 6:

[0137] Based on the local discrimination super-resolution reconstruction method based on multi-order infrared image degradation simulation in Embodiment 1, it further includes:

[0138] The user inputs the low-resolution infrared image to be processed into the trained super-resolution reconstruction network for processing to obtain a super-resolution reconstructed infrared image;

[0139] And send the super-resolution reconstructed infrared image to the corresponding user terminal for display based on the preset sending address.

[0140] Beneficial effects of the above technical solution: In the present invention, a user inputs a low-resolution infrared image to be processed into a trained super-resolution reconstruction network for processing to obtain a super-resolution reconstructed infrared image; and based on a preset sending address, the super-resolution reconstructed infrared image is sent to a corresponding client for display, realizing automatic reconstruction of the low-resolution infrared image and intelligent data transmission of the super-resolution reconstructed image.

[0141] Embodiment 7:

[0142] Based on Embodiment 1, a local discriminant super-resolution reconstruction method based on multi-order infrared image degradation simulation further includes:

[0143] Based on big data, a large number of high-resolution infrared images are obtained as original infrared images to establish a training database, and the output super-resolution reconstructed images corresponding to the trained super-resolution reconstruction network are collected to update the training database;

[0144] When the data update amount of the training database reaches the minimum update threshold, based on the latest training database, multiple original infrared images are randomly selected, and Steps 1-4 are executed.

[0145] Beneficial effects of the above technical solution: In the present invention, based on big data, a large number of high-resolution infrared images are obtained as original infrared images to establish a training database, which provides a data basis for the training of the model. At the same time, the output super-resolution reconstructed images corresponding to the trained super-resolution reconstruction network are collected to update the training database. When the data update amount of the training database reaches the minimum update threshold, based on the latest training database, multiple original infrared images are randomly selected to retrain the super-resolution reconstruction network, realizing the autonomous learning and regular update of the super-resolution reconstruction network, which is beneficial to improving the accuracy of image reconstruction.

[0146] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalent technologies, the present invention also intends to include these changes and modifications.

Claims

1. A local discriminant super-resolution reconstruction method based on multi-order infrared image degradation simulation, characterized in that: include: Step 1: Input the original infrared image into the multi-order infrared image degradation model to obtain a low-resolution degraded image; Step 2: Input the low-resolution degraded image into the generator of the super-resolution reconstruction network to generate a super-resolution reconstructed image; Step 3: After feature fusion of the super-resolution reconstruction and the original infrared image, the super-resolution reconstruction is input into the local discriminator of the super-resolution reconstruction network for processing, and the local discriminant loss is obtained; Step 4: Perform a weighted summation of the local discrimination loss and the preset model loss to obtain the image reconstruction training loss of the super-resolution reconstructed image; When the image reconstruction training loss is greater than the minimum loss value, return to step 2; Otherwise, the training is terminated and the trained super-resolution reconstruction network is output.

2. According to the local discrimination super-resolution reconstruction method based on multi-order infrared image degradation simulation according to claim 1, step 1 comprises: When the original infrared image is input into the multi-order infrared image degradation model, the multi-order infrared image degradation model randomly arranges a plurality of image degradation strategies based on the image degradation strategy database to generate a multi-order infrared image degradation process; Based on the multi-order infrared image degradation process, the original infrared image is processed to obtain a low-resolution degraded image; Wherein, the multi-order infrared image degradation model is a third-order degradation model.

3. According to the local discrimination super-resolution reconstruction method based on multi-order infrared image degradation simulation as described in claim 1, step 2 comprises: After the low-resolution infrared image is subjected to the first convolution processing and input, it is processed based on N RRDB modules to obtain a first output image; After performing a second convolution process on the first output image, upsampling is performed to obtain an upsampled image; Based on the residual dense network, the image features extracted by each network layer of the upsampled image and the low-resolution infrared image are respectively obtained to obtain the upsampled image features and the original image features of the low-resolution infrared image; The upsampled image features and the original image features are convolutionally fused to obtain a super-resolution reconstructed image.

4. According to the local discrimination super-resolution reconstruction method based on multi-order infrared image degradation simulation as described in claim 1, step 3 comprises: Based on a preset rectangular window, a non-overlapping slide is performed on the super-resolution reconstructed image, and a local entropy corresponding to the preset window is calculated to determine whether the local entropy is greater than a local entropy threshold; If yes, based on the error value between the local entropy and the local entropy threshold, the rectangular window is adaptively adjusted, and the pixel point data in the adjusted rectangular window is compared and calculated with the pixel point data corresponding to the original infrared image to obtain a local residual map; Otherwise, the pixel point data within the preset rectangular window is obtained and compared with the pixel point data corresponding to the original infrared image to obtain a local residual map; Based on the local residual map, calculate the local variance of each pixel point in the adjusted rectangular window or the preset rectangular frame, and when the local variance is greater than the preset variance, determine that the current pixel point is an artifact pixel point; Otherwise, the current pixel is determined to be a true shadow pixel; Based on the local variance corresponding to all artifact pixels in the super-resolution reconstructed image, the local discrimination loss is calculated.

5. The local discrimination super-resolution reconstruction method based on multi-order infrared image degradation simulation according to claim 4 is characterized in that: Based on the error value between the local entropy and the local entropy threshold, the rectangular window is adaptively adjusted, including: Based on big data technology, the window information entropy corresponding to the infrared images of rectangular windows of different sizes is collected; The window information entropy is grouped based on the window size to obtain rectangular window data groups of multiple sizes, a window information entropy range corresponding to each rectangular window data group is determined, and whether there is an overlapping range of window information entropy ranges corresponding to rectangular windows of different sizes is determined; If it does not exist, the window information entropy range corresponding to the current rectangular windows of different sizes is used as the final window information entropy range; If so, obtaining rectangular window data groups corresponding to the overlapping rectangular windows as the first target data group and the second target data group, respectively, clustering the data in the first target data group and the second target data group, and obtaining the first cluster center and the second cluster center; Based on the first cluster center and the second cluster center, determine the target window information entropy value corresponding to the distance center point, and judge whether the target window information entropy value is within the overlapping range. If so, correct the upper limit value of the window information entropy range corresponding to the first target data group and the lower limit value of the window information entropy range corresponding to the second target data group based on the target window information entropy value; Otherwise, when the target window information entropy value is less than or equal to the lower limit value of the overlapping range, the lower limit value of the window information entropy range corresponding to the second target data group is corrected based on the lower limit value of the overlapping range; When the target window information entropy value is greater than or equal to the upper limit value of the overlapping range, the lower limit value of the window information entropy range corresponding to the first target data group is corrected based on the upper limit value of the overlapping range; Based on the correction results, the final window information entropy range corresponding to rectangular windows of different sizes is determined; According to the final window information entropy range corresponding to the rectangular windows of different sizes, the window information entropy difference interval between the rectangular windows of different sizes is determined; The error value between the local entropy and the local entropy threshold is matched with the window information entropy difference interval between rectangular windows of multiple sizes and the preset rectangular window to determine the size corresponding to the optimal rectangular window, and the window size of the preset rectangular window is adaptively adjusted according to the size corresponding to the optimal rectangular window.

6. According to claim 5, the local discrimination super-resolution reconstruction method based on multi-order infrared image degradation simulation calculates the local discrimination loss based on the local variance corresponding to all artifact pixels in the super-resolution reconstructed image, comprising: The standard local discrimination loss corresponding to the super-resolution reconstructed image is calculated based on a preset local discrimination loss model, and the artifact pixels in the super-resolution reconstructed image are marked, and the distribution characteristics of the artifact pixels are extracted based on the marking results; Based on the distribution characteristics, each image element in the super-resolution reconstructed image is evaluated, and according to the evaluation result, the image distortion rate of the super-resolution reconstructed image is determined; The reconstruction global error of the super-resolution reconstructed image is evaluated based on the image distortion rate, and when the reconstruction error is greater than a preset error, a reconstruction global error index is obtained based on an error rate between the reconstruction error and the preset error, and a standard local discrimination error is corrected based on the reconstruction global error index to obtain an output local discrimination error; Otherwise, the standard local discriminant error is used as the output local discriminant error.

7. According to claim 6, a local discrimination super-resolution reconstruction method based on multi-order infrared image degradation simulation is used to evaluate each image element in the super-resolution reconstructed image based on distribution characteristics, including: Perform edge detection on the super-resolution reconstructed image, and based on the edge detection result, divide the image information on the super-resolution reconstructed image into multiple image element information; Based on the distribution characteristics of artifact pixels and the image element information segmentation results, the correlation relationship between the image representation information of each artifact image on the effective image is determined; Based on the association relationship, determine the number of artifact pixels and position distribution sub-features on each image element; Obtaining the total number of pixels corresponding to each image element having an artifact pixel point respectively, and obtaining an artifact index of the corresponding image element according to the total number of pixels and the number of artifact pixels of the corresponding image element; When the artifact index is greater than or equal to a preset threshold, determining that the corresponding image element is distorted; Otherwise, based on the position distribution sub-features corresponding to the artifact pixels on the corresponding image element, determine whether there is an artifact pixel concentration area on the corresponding image element. If so, obtain the element distortion ratio based on the ratio of the number of artifact pixels in the artifact pixel concentration area to the total number of pixels corresponding to the corresponding image element. Based on the element distortion ratio, determining the maximum artifact pixel concentration area, when the element distortion ratio corresponding to the maximum artifact pixel concentration area is greater than or equal to the threshold distortion ratio, determining that the corresponding image element is distorted; Otherwise, it is determined that the corresponding image element is not distorted.

8. According to the local discrimination super-resolution reconstruction method based on multi-order infrared image degradation simulation of claim 7, determining the image distortion rate of the super-resolution reconstructed image according to the evaluation result, comprising: Traversing the image elements on the super-resolution reconstructed image to determine the total number of image elements and the number of distorted image elements corresponding to the super-resolution reconstructed image; Based on the total number of image elements corresponding to the super-resolution reconstructed image and the number of distorted image elements, a distortion element ratio of the super-resolution reconstructed image is obtained, and the distortion element ratio is used as the image distortion rate of the super-resolution reconstructed image.

9. The local discrimination super-resolution reconstruction method based on multi-order infrared image degradation simulation according to claim 1 is characterized in that: Also includes: The user inputs the low-resolution infrared image to be processed into the trained super-resolution reconstruction network for processing to obtain a super-resolution reconstructed infrared image; And based on a preset sending address, the super-resolution reconstructed infrared image is sent to a corresponding user terminal for display.

10. The local discrimination super-resolution reconstruction method based on multi-order infrared image degradation simulation according to claim 9, characterized in that: Also includes: Based on big data, a large number of high-resolution infrared images are obtained as original infrared images to establish a training database, and the output super-resolution reconstructed images corresponding to the trained super-resolution reconstruction network are collected to update the training database; When the data update amount of the training database reaches the minimum update threshold, a plurality of original infrared images are randomly selected based on the latest training database, and steps 1-4 are executed.

Images