Oversampling image super-resolution reconstruction method and device for one-dimensional linear push-broom imaging

Abstract

The invention discloses an oversampling image super-resolution reconstruction method and device for one-dimensional linear push-broom imaging, and belongs to the field of remote sensing image processing. The method comprises the following steps: calculating the effective information amount of an oversampling image of one-dimensional linear push-broom imaging so as to determine a super-resolution multiple; based on the super-resolution multiple, determining a pixel gray mapping equation between the super-resolution image to be reconstructed and the oversampling image; and solving the pixel gray mapping equation by using a least square method so as to determine the optimal gray value matching by using the sum of squares of the minimum error, and reconstructing to obtain a super-resolution image. According to the scheme, oversampling image super-resolution reconstruction under the condition of one-dimensional linear push-broom can be realized, redundant information gain brought by oversampling can be well utilized, and a reconstructed image with higher precision and clearer is obtained.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing image processing, and particularly relates to a method and device for super-resolution reconstruction of oversampled images in one-dimensional linear pushbroom imaging. Background Art

[0002] The super-resolution reconstruction technology of oversampled images mainly solves the problems of improving imaging resolution and compensating for the lack of spatial information caused by factors such as terrain undulation or sensor limitations. In the field of remote sensing imaging, the one-dimensional linear pushbroom imaging method is mostly used. Compared with the two-dimensional imaging method, the one-dimensional linear pushbroom imaging method has higher spatial resolution and higher geometric accuracy, and is more suitable for capturing fast-moving objects.

[0003] Since one-dimensional linear pushbroom can increase the sampling frequency by controlling the sensor swing, thereby achieving oversampling of the ground background. For traditional one-dimensional linear pushbroom imaging images, linear and bilinear interpolation methods are commonly used to implement the reconstruction of oversampled images. However, interpolation methods mainly focus on the smoothing and interpolation between pixels, lacking in-depth understanding of complex structures and textures in the image, resulting in poor performance of the reconstructed image. Due to the slow change of the background gray level in large-scale remote sensing images, artifacts are easily generated, the effect is not clear enough, and the redundant information gain brought by oversampling is lost.

[0004] Therefore, there is an urgent need to provide a new method for super-resolution reconstruction of oversampled images in one-dimensional linear pushbroom imaging. Summary of the Invention

[0005] In order to solve the problem that the commonly used interpolation method in one-dimensional linear pushbroom imaging for reconstructing oversampled images results in poor performance of the reconstructed image and unclear effect, the embodiments of the present invention provide a method and device for super-resolution reconstruction of oversampled images in one-dimensional linear pushbroom imaging.

[0006] On the one hand, a method for super-resolution reconstruction of oversampled images in one-dimensional linear pushbroom imaging is provided. The method includes:

[0007] Calculating the effective information amount of the oversampled image in one-dimensional linear pushbroom imaging to determine the super-resolution multiple;

[0008] Based on the super-resolution multiple, determining the pixel gray-level mapping equation between the super-resolution image to be reconstructed and the oversampled image;

[0009] Using the least squares method to solve the pixel gray-level mapping equation to determine the best gray-level value matching by minimizing the sum of squared errors, and reconstructing the super-resolution image.

[0010] On the other hand, a device for super-resolution reconstruction of oversampled images in one-dimensional linear pushbroom imaging is provided for implementing the steps described in any method embodiment of the specification. The device includes:

[0011] A calculation unit for calculating the effective information amount of an oversampled image of one-dimensional linear pushbroom imaging to determine the super-resolution multiple;

[0012] A mapping unit for determining a pixel gray-level mapping equation between the super-resolution image to be reconstructed and the oversampled image based on the super-resolution multiple;

[0013] A solving unit for solving the pixel gray-level mapping equation by using the least squares method to determine the best gray-level value matching by minimizing the sum of squared errors and reconstructing the super-resolution image.

[0014] On the other hand, a computer device is provided. The computer device includes a memory and a processor. The memory is used to store a computer program, and the processor is used to execute the computer program stored on the memory to implement the steps of the above method.

[0015] On the other hand, a computer-readable storage medium is provided. The storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0016] On the other hand, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0017] The technical solution provided by the present invention can at least bring the following beneficial effects:

[0018] In view of the oversampling situation generated in linear pushbroom imaging, the super-resolution multiple is determined by considering the effective information amount of the oversampled image. An equation is established for the pixel gray-level relationship between the oversampled image and the super-resolution image, and the super-resolution image is obtained by using the least squares method. In this way, the super-resolution reconstruction of the oversampled image in the case of one-dimensional linear pushbroom is realized, and the redundant information gain brought by oversampling is well utilized to obtain a reconstructed image with higher accuracy and clarity. This solution has the advantages of optimizing the availability of remote sensing data, enhancing the interpretability of remote sensing data, and better meeting the needs of users. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0020] Figure 1It is a flowchart of a super-resolution reconstruction method for oversampled images of one-dimensional linear pushbroom imaging provided by an embodiment of the present invention;

[0021] Figure 2 It is a structural diagram of a device for super-resolution reconstruction of oversampled images of one-dimensional linear pushbroom imaging provided by an embodiment of the present invention;

[0022] Figure 3 It is a hardware architecture diagram of a computer device provided by an embodiment of the present invention. Specific Embodiments

[0023] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0024] The following describes the specific implementation manners of the above concepts.

[0025] Please refer to Figure 1 , a super-resolution reconstruction method for oversampled images of one-dimensional linear pushbroom imaging provided by an embodiment of the present invention, the method includes:

[0026] Step 100: Calculate the effective information amount of the oversampled image of one-dimensional linear pushbroom imaging to determine the super-resolution multiple;

[0027] Step 102: Based on the super-resolution multiple, determine the pixel gray-level mapping equation between the super-resolution image to be reconstructed and the oversampled image;

[0028] Step 104: Use the least squares method to solve the pixel gray-level mapping equation to determine the best gray-level value matching by minimizing the sum of squared errors, and reconstruct the super-resolution image.

[0029] In the embodiments of the present invention, for the oversampling situation generated in linear pushbroom imaging, the super-resolution multiple is determined by considering the effective information amount of the oversampled image. An equation is established for the pixel gray-level relationship between the oversampled image and the super-resolution image, and the super-resolution image is obtained by using the least squares method, so as to realize the super-resolution reconstruction of the oversampled image in the case of one-dimensional linear pushbroom, make good use of the redundant information gain brought by oversampling, and obtain a reconstructed image with higher accuracy and clearer. This solution has the advantages of optimizing the usability of remote sensing data, enhancing the interpretability of remote sensing data, and better meeting user needs.

[0030] The following describes Figure 1The execution manner of each step shown

[0031] For step 100:

[0032] In some embodiments, step 100 may include:

[0033] Determine the effective information volume based on the resolution and oversampling rate of the oversampled image of one-dimensional linear pushbroom imaging;

[0034] Determine the super-resolution multiple based on the effective information volume and the resolution of the oversampled image.

[0035] In this embodiment, for the field of remote sensing of linear pushbroom imaging, it is possible to control the swing scan to achieve oversampled pushbroom imaging, thereby introducing gain information of imaging. To achieve super-resolution reconstruction of the oversampled image, it is necessary to define and quantify the effective information volume of the oversampled image, so as to use the effective information volume to evaluate the information density of the super-resolution image and determine the super-resolution multiple.

[0036] In some embodiments, the effective information volume is calculated in the following manner:

[0037]

[0038] In the formula, I is the effective information volume, R m is the resolution of the oversampled image, and n is the oversampling rate.

[0039] In this embodiment, the effective information volume can be defined according to the above formula using the resolution and oversampling rate of the oversampled image.

[0040] In some embodiments, the super-resolution multiple is calculated in the following manner:

[0041]

[0042] In the formula, k is the super-resolution multiple, R m is the resolution of the oversampled image.

[0043] In this embodiment, the maximum super-resolution multiple that can be supported can be determined according to the above formula using the effective information volume. It can be understood that taking the maximum value supported by the super-resolution multiple can make the resolution of the super-resolution image the largest.

[0044] For step 102:

[0045] In some embodiments, step 102 may include:

[0046] Determine the size of the super-resolution image to be reconstructed based on the super-resolution multiple and the resolution of the oversampled image;

[0047] Based on the resolution of the oversampled image and the size of the super-resolution image to be reconstructed, a pixel gray-level mapping equation between the super-resolution image to be reconstructed and the oversampled image is established.

[0048] In this embodiment, first, the size of the super-resolution image to be reconstructed is determined by using the super-resolution multiple, and then the position and gray-level relationship of pixels between the super-resolution image to be reconstructed and the oversampled image, that is, the pixel gray-level mapping equation, is established to map and reconstruct the super-resolution image.

[0049] The pixel gray-level mapping equation is:

[0050] Ax = b

[0051] In the formula, A is the coefficient matrix, x is the super-resolution image, and b is the oversampled image.

[0052] Regarding step 104:

[0053] In this step, according to the physical principle of super-resolution imaging, it can be known that the pixel gray-level mapping equation obtained is an ill-conditioned equation. The least squares method can be selected to find the best super-resolution image gray-level value matching by minimizing the sum of the squares of the errors, and the super-resolution image is reconstructed. Then the formula of the least squares method is:

[0054] x = (A T A) -1 A T b

[0055] Among them, x is the super-resolution image, b is the oversampled image, and A is the coefficient matrix.

[0056] For the high-resolution image obtained by using the above method, the super-resolution reconstruction of the oversampled image is initially realized, but it is less sensitive to noise, texture, etc. Accordingly, on this basis, the processed high-resolution image can be used as a relatively accurate initial reference image, and the projection onto convex sets method is introduced to further correct the error with the potential true image.

[0057] The projection onto convex sets method uses the spatial response characteristics and noise characteristics of the imaging system as prior knowledge, and takes them as the constraint conditions for image reconstruction. Each constraint condition corresponds to a convex set containing the ideal high-resolution image in the entire imaging space. Any point in the intersection of these convex sets is considered to be an acceptable image reconstruction result. This algorithm is a process of starting from an arbitrary point in space and projecting and positioning it onto the intersection of the convex sets. Computationally, this algorithm adopts an iterative correction method. Starting from a point in a convex set, the position of this point is corrected using the restrictions of other sets until a point in the intersection is located. Generally speaking, the point projected onto the intersection is not unique, so the final result is often related to the selection of the initial value.

[0058] Therefore, after obtaining the super-resolution image by using the least squares method, it may further include:

[0059] Taking the super-resolution image as the high-resolution image in the first iteration of the convex set projection method, and taking the oversampled image as the low-resolution image in the first iteration;

[0060] Using the imaging model and the residual correction model of the convex set projection method for iteration to perform secondary correction on the super-resolution image to obtain the final super-resolution image.

[0061] In this embodiment, the imaging model and the residual correction model of the convex set projection method are respectively expressed as:

[0062] g(x,y) = Σ p,q f(p,q)h(y,q) + n(x,y)

[0063]

[0064] In the formula, x and y are the pixel positions of the low-resolution image g, p and q are the pixel positions of the high-resolution image f, h is the point spread function of the system, n is the additive noise of the system, r is the residual, and σ is the threshold of the residual.

[0065] In this embodiment, taking the obtained super-resolution image with higher accuracy as the high-resolution image in the first iteration of the convex set projection method, migrating the convex set projection method to the imaging reconstruction of linear pushbroom, and degrading the Gaussian point spread function introduced by the traditional method into the blur in the pushbroom direction. Therefore, the point spread function h degenerates from two-dimensional to one-dimensional. Using the Laplace operator to extract the additive noise characteristics in the low-resolution oversampled pushbroom image, and performing iteration by using the imaging model and the residual correction model of the convex set projection method to perform secondary correction on the super-resolution image to obtain the final super-resolution image, which can further improve the accuracy and clarity of the reconstructed image.

[0066] In summary, in view of the oversampling situation occurring in linear pushbroom imaging, the present invention determines the super-resolution multiple by considering the effective information amount of oversampling, uses the least squares method as the initialization reference of the reconstruction algorithm, and introduces the convex set projection method to finally correct the super-resolution image solved by the least squares method, realizing the super-resolution reconstruction of the oversampled image in one-dimensional linear pushbroom imaging, and having the advantages of optimizing the usability of remote sensing data, enhancing the interpretability of remote sensing data, and better meeting the user requirements.

[0067] Please refer to Figure 2 , the embodiment of the present invention provides an oversampled image super-resolution reconstruction device for one-dimensional linear pushbroom imaging, which is used to implement the steps of any method embodiment in the specification. The device includes:

[0068] The calculation unit 201 is configured to calculate the effective information amount of the oversampled image of the one-dimensional linear pushbroom imaging to determine the super-resolution multiple;

[0069] The mapping unit 202 is configured to determine the pixel gray-scale mapping equation between the super-resolution image to be reconstructed and the oversampled image based on the super-resolution multiple;

[0070] The solving unit 203 is configured to solve the pixel gray-scale mapping equation by using the least squares method to determine the best gray-scale value matching by minimizing the sum of the squares of the errors, and reconstruct the super-resolution image.

[0071] In an embodiment of the present invention, the calculation unit 201 is configured to execute:

[0072] Determine the effective information amount based on the resolution and oversampling rate of the oversampled image of the one-dimensional linear pushbroom imaging;

[0073] Determine the super-resolution multiple based on the effective information amount and the resolution of the oversampled image.

[0074] In an embodiment of the present invention, the effective information amount in the calculation unit 201 is calculated by the following method:

[0075]

[0076] In the formula, I is the effective information amount, and R m is the resolution of the oversampled image, and n is the oversampling rate.

[0077] In an embodiment of the present invention, the super-resolution multiple in the calculation unit 201 is calculated by the following method:

[0078]

[0079] In the formula, k is the super-resolution multiple, and R m is the resolution of the oversampled image.

[0080] In an embodiment of the present invention, the mapping unit 202 is configured to execute:

[0081] Determine the size of the super-resolution image to be reconstructed based on the super-resolution multiple and the resolution of the oversampled image;

[0082] Establish the pixel gray-scale mapping equation between the super-resolution image to be reconstructed and the oversampled image based on the resolution of the oversampled image and the size of the super-resolution image to be reconstructed.

[0083] In an embodiment of the present invention, after the solving unit 203, there is further a correction unit 204 configured to execute:

[0084] Take the super-resolution image as the high-resolution image in the first iteration of the convex set projection method, and take the oversampled image as the low-resolution image in the first iteration;

[0085] Use the imaging model and residual correction model of the convex set projection method for iteration to perform secondary correction on the super-resolution image to obtain the final super-resolution image.

[0086] It should be noted that: the oversampled image super-resolution reconstruction device for one-dimensional linear pushbroom imaging provided in the above embodiments is only illustrated by dividing the above functional units. In practical applications, the above functions can be allocated to different functional units according to needs, that is, the internal structure of the device is divided into different functional units to complete all or part of the functions described above. In addition, the above device embodiments and method embodiments belong to the same concept, and the specific implementation process can be seen in the method embodiments, which will not be elaborated here.

[0087] The embodiments of the present application also provide a computer device. Please refer to Figure 3 which includes a processor and a memory. At least one instruction, at least one program, a code set or an instruction set is stored in the memory, and at least one instruction, at least one program, a code set or an instruction set is loaded and executed by the processor to implement the oversampled image super-resolution reconstruction method for one-dimensional linear pushbroom imaging provided in the above method embodiments.

[0088] The embodiments of the present application also provide a computer-readable storage medium. At least one instruction, at least one program, a code set or an instruction set is stored on the computer-readable storage medium, and at least one instruction, at least one program, a code set or an instruction set is loaded and executed by the processor to implement the oversampled image super-resolution reconstruction method for one-dimensional linear pushbroom imaging provided in the above method embodiments.

[0089] The embodiments of the present application also provide a computer program product. The computer program product includes a computer program. The processor of the computer device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the computer device executes the oversampled image super-resolution reconstruction method for one-dimensional linear pushbroom imaging in any of the above embodiments.

[0090] For the convenience of description, when describing the above system or device, various modules or units are described separately according to functions. Of course, when implementing the present application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0091] As can be seen from the description of the above embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of this application, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disc, etc., and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of various embodiments or some parts of the embodiments of this application.

[0092] Finally, it should also be noted that in this text, relational terms such as first, second, third, and fourth are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. Without more limitations, an element defined by the statement "including a..." does not exclude the existence of additional identical elements in the process, method, article or device including the element.

[0093] The above are only the preferred embodiments of this application. It should be pointed out that for those of ordinary skill in the art, without departing from the principle of this application, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of this application.

Claims

1. A method for super-resolution reconstruction of oversampled images for one-dimensional linear push-scan imaging, characterized in that: include: Calculate the effective information content of the oversampled image of one-dimensional linear push-scan imaging to determine the super-resolution multiple; Based on the super-resolution multiple, determining a pixel grayscale mapping equation between the super-resolution image to be reconstructed and the oversampled image; The pixel grayscale mapping equation is solved by using the least square method to determine the best grayscale value match by minimizing the sum of squares of errors, and a super-resolution image is reconstructed.

2. The method according to claim 1, characterized in that The calculating the effective information amount of the oversampled image of the one-dimensional linear push-scan imaging to determine the super-resolution multiple includes: Determine the effective information volume based on the resolution and oversampling ratio of the oversampled image of one-dimensional linear push-scan imaging; A super-resolution factor is determined based on the effective information amount and the resolution of the oversampled image.

3. The method according to claim 2, characterized in that The effective amount of information is calculated as follows: In the formula, I is the effective information amount, R m is the resolution of the oversampled image, and n is the oversampling rate.

4. The method according to claim 2, characterized in that The super-resolution multiple is calculated as follows: In the formula, k is the super-resolution multiple, R m is the resolution of the oversampled image.

5. The method according to claim 1, characterized in that Based on the super-resolution multiple, determining a pixel grayscale mapping equation between the super-resolution image to be reconstructed and the oversampled image includes: Determining the size of the super-resolution image to be reconstructed based on the super-resolution multiple and the resolution of the oversampled image; Based on the resolution of the oversampled image and the size of the super-resolution image to be reconstructed, a pixel grayscale mapping equation between the super-resolution image to be reconstructed and the oversampled image is established.

6. The method according to claim 1, characterized in that After obtaining the super-resolution image, it also includes: Using the super-resolution image as a high-resolution image at the first iteration of the convex set projection method, and using the over-sampled image as a low-resolution image at the first iteration; The imaging model of the convex set projection method and the residual correction model are iterated to perform secondary correction on the super-resolution image to obtain a final super-resolution image.

7. A one-dimensional linear push-scan imaging oversampled image super-resolution reconstruction device, used to implement the steps of any of the methods described in claims 1-6, characterized in that: include: A calculation unit, used for calculating the effective information amount of the oversampled image of the one-dimensional linear push-scan imaging to determine the super-resolution multiple; A mapping unit, used to determine a pixel grayscale mapping equation between a super-resolution image to be reconstructed and the oversampled image based on the super-resolution multiple; The solving unit is used to solve the pixel grayscale mapping equation by using the least square method, so as to determine the best grayscale value match by minimizing the sum of squares of errors, and reconstruct a super-resolution image.

8. A computer device, characterized in that: The computer device includes a memory and a processor, the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory to implement the steps of any one of the methods described in claims 1-6.

9. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method described in any one of claims 1 to 6 are implemented.

10. A computer program product, characterized in that The method comprises a computer program, wherein when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images