Critical diffraction limit ultra-large width high-resolution optical remote sensing imaging method

Abstract

The invention discloses a critical diffraction limit ultra-large width high-resolution optical remote sensing imaging method. The method comprises the following steps: S1, arranging a sub-pixel dynamic coding plate on the surface of a TDI CMOS (Time Delay Integration Complementary Metal-Oxide-Semiconductor Transistor) device; s2, sampling the low-resolution image through a TDI CMOS device to obtain a frame of coded image; s3, replacing different sub-pixel dynamic coding plates, and repeating the steps S1 to S2 to obtain multiple frames of coding images; and S4, reconstructing the multi-frame coded image to obtain a high-resolution image. According to the invention, the technical bottleneck of improving the resolution of the ultra-wide optical remote sensing camera is broken through.

Description

Technical Field

[0001] The invention belongs to the technical field of aerospace optical remote sensing, and in particular relates to a critical diffraction limit ultra-wide width high-resolution optical remote sensing imaging method. Background Art

[0002] The space-bandwidth product is used to describe the information transmission capability of an imaging system. The space-bandwidth product of an optical imaging system is equal to the product of the effective field of view and the passband area determined by the system cutoff frequency. The system space-bandwidth product is invariant. The larger the system space-bandwidth product, the more difficult it is to design and manufacture. According to the space-bandwidth product theory, there is an upper limit to the information acquisition capability of an optical remote sensing camera system. The broadening of the signal acquired by the optical camera system in the spatial domain and the frequency domain is mutually restricted. The width and resolution cannot be achieved at the same time. The traditional optical system is subject to the long focal length requirement of the remote sensing camera (high spatial resolution), and the achievable field of view of the optical camera is not very large.

[0003] Reducing the size of detector pixels can increase the pixel scale of single-chip devices, and can also reduce the volume, weight and power consumption of the entire optical imaging system, thereby greatly saving costs. However, small-pixel detector devices need to be matched with high-quality optical lenses with high Nyquist frequencies, which poses new challenges to traditional optical design. Traditional optical design improves image quality at high Nyquist frequencies by modifying aspheric reflectors into high-order free-form surfaces or adding correction mirrors, which greatly increases system complexity, processing difficulty, and assembly difficulty, requiring a large price to be paid.

[0004] Due to the difficulty in engineering large optical components, the diffraction-limited cutoff frequency of optical systems, and the boundary limitations of traditional system design methods, the cost of developing higher-resolution and wider-width Earth observation optical remote sensing cameras is becoming increasingly high.

[0005] In summary, there are serious technical bottlenecks in improving the resolution capabilities of ultra-wide-width optical remote sensing cameras. It is difficult for traditional optical designs to achieve the 0.25m spatial resolution and 150km width optical imaging system indicators. Summary of the invention

[0006] The technical problem solved by the present invention is: to overcome the deficiencies of the prior art, to provide a critical diffraction limit ultra-wide-width high-resolution optical remote sensing imaging method, and to break through the technical bottleneck of improving the resolution of ultra-wide-width optical remote sensing cameras.

[0007] The object of the present invention is achieved through the following technical scheme: a critical diffraction limit ultra-wide width high-resolution optical remote sensing imaging method, comprising: step S1: setting a sub-pixel dynamic coding plate on the surface of a TDI CMOS device; step S2: sampling a low-resolution image through the TDI CMOS device to obtain a frame of coded image; step S3: replacing different sub-pixel dynamic coding plates, repeating steps S1 to S2, and obtaining multiple frames of coded images; step S4: reconstructing multiple frames of coded images to obtain a high-resolution image.

[0008] In the above critical diffraction limit ultra-wide width high-resolution optical remote sensing imaging method, the pixel size of the TDI CMOS device is 7um, and the pixel size of the sub-pixel dynamic encoding plate is 3.5um.

[0009] In the above critical diffraction limit ultra-wideband high-resolution optical remote sensing imaging method, each pixel of the TDI CMOS device corresponds to a pixel of a 2×2 sub-pixel dynamic encoding plate.

[0010] In the above critical diffraction limit ultra-wide width high-resolution optical remote sensing imaging method, in the sub-pixel dynamic coding plate, the sub-pixel coding of the first two columns is the same as the sub-pixel coding of the adjacent following two columns.

[0011] In the above-mentioned critical diffraction limit ultra-wide width high-resolution optical remote sensing imaging method, the encoding method of each sub-pixel dynamic coding plate includes: randomly generating W groups of coding matrices, where W is a positive integer; solving the condition number of the coding matrix; sorting the condition number of the coding matrix from small to large to obtain a serial number; performing cross-correlation calculation on the coding matrix corresponding to each serial number to obtain a coding matrix with the minimum cross-correlation equal to the number of sub-pixel dynamic coding plates, and assigning each coding matrix with the minimum cross-correlation to the corresponding sub-pixel dynamic coding plate.

[0012] In the above critical diffraction limit ultra-wideband high-resolution optical remote sensing imaging method, W ≥ 1000.

[0013] In the above critical diffraction limit ultra-wide width high-resolution optical remote sensing imaging method, the value of the encoding matrix is ​​[0,1].

[0014] In the above critical diffraction limit ultra-wideband high-resolution optical remote sensing imaging method, the condition number of the encoding matrix is ​​obtained by the following formula:

[0015]

[0016] ∑1=diag(σ1,σ2,…,σ r );

[0017] Among them, C i is the condition number of the i-th group of encoding matrices, A iis the i-th group encoding matrix, min(·) represents the minimum value operator, max(·) represents the maximum value operator, is the encoding matrix A i The diagonal matrix after singular value decomposition, ∑1 is the non-negative real number on the diagonal, σ1 is the encoding matrix A i The first singular value of , σ2 is the encoding matrix A i The second singular value of r is the encoding matrix A i The rth singular value of i The number of non-zero singular values.

[0018] A critical diffraction limit ultra-wide width high-resolution optical remote sensing imaging device comprises: a first module for setting a sub-pixel dynamic coding plate on the surface of a TDI CMOS device; a second module for sampling a low-resolution image through the TDI CMOS device to obtain a frame of coded image; a third module for replacing different sub-pixel dynamic coding plates to obtain multiple frames of coded images; and a fourth module for reconstructing multiple frames of coded images to obtain a high-resolution image.

[0019] An electronic device comprises: a memory for storing computer-readable instructions; and a processor for running the computer-readable instructions to execute a critical diffraction limit ultra-wide width high-resolution optical remote sensing imaging method.

[0020] Compared with the prior art, the present invention has the following beneficial effects:

[0021] The present invention only needs to cut in (cut out) the sub-pixel dynamic coding plate in front of the focal plane to achieve an improvement in resolution, breaking through the technical bottleneck of improving the resolution of ultra-wide-width optical remote sensing cameras, with low engineering implementation difficulty and strong scalability. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0023] Figure 1 is a schematic diagram of an optical dynamic coding super-resolution imaging link provided by an embodiment of the present invention;

[0024] Figure 2 is a schematic diagram of the layout of the encoding plate provided by an embodiment of the present invention in an imaging system;

[0025] FIG3( a ) is a schematic diagram of an optical dynamic encoding method of a linear array TDI CMOS detector after adding an optical encoding plate according to an embodiment of the present invention;

[0026] FIG3( b ) is a schematic diagram of an optical dynamic encoding method of a linear array TDI CMOS detector before adding an optical encoding plate according to an embodiment of the present invention;

[0027] FIG4( a ) is a local image block correspondence diagram provided by an embodiment of the present invention;

[0028] FIG4( b ) is a schematic diagram of a coding matrix structure provided in an embodiment of the present invention;

[0029] Figure 5 It is a flow chart of optical dynamic coding imaging simulation and reconstruction of a linear array detector provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0030] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present disclosure and to be able to fully convey the scope of the present disclosure to those skilled in the art. It should be noted that, in the absence of conflict, the embodiments of the present invention and the features described in the embodiments can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0031] Critical diffraction-limited ultra-wideband high-resolution optical remote sensing imaging system, such as Figure 1 As shown, the optical dynamic coding plate is cut into the focal plane, and N 0.5m resolution remote sensing images with different codes are output. After being transmitted to the ground, super-resolution reconstruction is performed using the N optical coding images and the 0.25m optical coding plate calibration data, and then image quality improvement processing is performed to finally obtain the final image product.

[0032] This embodiment provides a critical diffraction limit ultra-wide width high-resolution optical remote sensing imaging method, the method comprising:

[0033] Step S1: placing a sub-pixel dynamic coding plate on the surface of a TDI CMOS device;

[0034] Step S2: sampling the low-resolution image through the TDI CMOS device to obtain a frame of coded image;

[0035] Step S3: Replace a different sub-pixel dynamic coding board, repeat steps S1 to S2, and obtain multiple frames of coded images;

[0036] Step S4: reconstruct multiple frames of coded images to obtain high-resolution images.

[0037] The pixel size of the TDI CMOS device is 7um, and the pixel size of the sub-pixel dynamic encoding plate is 3.5um. Each pixel of the TDI CMOS device corresponds to 2×2 pixels of the sub-pixel dynamic encoding plate.

[0038] In the sub-pixel dynamic coding board, the sub-pixel coding of the first two columns is the same as the sub-pixel coding of the adjacent two subsequent columns.

[0039] The encoding method of each sub-pixel dynamic coding plate includes: randomly generating W groups of coding matrices, where W is a positive integer; solving the condition number of the coding matrix; sorting the condition number of the coding matrix from small to large to obtain a serial number; performing cross-correlation calculation on the coding matrix corresponding to each serial number to obtain a coding matrix with the minimum cross-correlation equal to the number of sub-pixel dynamic coding plates, and assigning each coding matrix with the minimum cross-correlation to the corresponding sub-pixel dynamic coding plate.

[0040] W ≥ 1000. The value of the encoding matrix is ​​[0, 1].

[0041] The condition number of the encoding matrix is ​​obtained by the following formula:

[0042]

[0043] ∑1=diag(σ1,σ2,…,σ r );

[0044] Among them, C i is the condition number of the i-th group of encoding matrices, A i is the i-th group encoding matrix, min(·) represents the minimum value operator, max(·) represents the maximum value operator, is the encoding matrix A i The diagonal matrix after singular value decomposition is Σ1 is a non-negative real number on the diagonal, σ1 is the encoding matrix A i The first singular value of , σ2 is the encoding matrix A i The second singular value of r is the encoding matrix A i The rth singular value of i The number of non-zero singular values.

[0045] The sub-pixel dynamic encoding plate of this embodiment is integrated closely to the surface of the TDI CMOS device. The 7um single pixel corresponds to the pixel in the 2×23.5um sub-pixel optical encoding plate. The TDI CMOS device performs 32 levels of identical encoding on the same object, and accumulates the signals to output the linear array image of the first encoding. The next 32-level TDI CMOS area uses a different encoding form to collect the linear array image of the same object, and so on, collecting 8 linear array images with different encodings (i.e., multi-frame encoded images).

[0046] In the sub-pixel dynamic coding board, in the first 32-level coding, the first two columns of sub-pixel coding are the same as the coding of the last two columns, and the same coding can be performed on the same object. After integration and accumulation, the images corresponding to the two columns of sub-pixel coding in the first 32-level coding are sensed by one column of detectors; the first two columns of sub-pixel coding in the first 32-level coding are completely different from the first two columns of sub-pixel coding in the N-1th and Nth 32-level coding, so there are N different codes. Sub-pixel coding sampling is performed on the same column of objects through push-scanning to achieve optical dynamic coding.

[0047] The coding optimization method is a random non-coherent local optimal coding optimization method. The details are as follows:

[0048] Step 1: Randomly generate W groups (W ≥ 1000) of encoding matrices A with values ​​of [0, 1] i (i=1,2,…,W), the encoding matrix size is N×B pixels;

[0049] Step 2: Solve each encoding matrix A i Condition number C of (i=1,2,…,W) i (i=1,2,…,W):

[0050]

[0051] Among them, A i is the random coding matrix of the i-th linear array, min(·) represents the minimum operator, max(·) represents the maximum operator, U and V are singular values ​​σ i The left and right singular vectors of ∑1=diag(σ1,σ2,…,σ r );

[0052] Step 3: C i (i=1,2,…,W) are sorted from small to large and stored in sequence number K i (i=1,2,…,W);

[0053] Step 4: Assume that j is in [K1,K int[W·P]], where P = 10%, int[·] represents an integer operator, then the N (N < int[W·P]) coding matrices with the smallest mutual correlation are calculated according to the following formula:

[0054]

[0055] Among them, E[] is the average value operation, is the product operation of variable j from 1 to N, j is the variable, K1 is the first value of array K, K int[W·P] Take the value of int[W·P]th in the array K.

[0056] This embodiment uses an optimized optical dynamic coding matrix group to perform pixel subdivision to achieve sub-pixel sampling, and realizes super-resolution imaging of the TDI CMOS device imaging system through partitioned sub-pixel coding technology; local compressed sensing technology is used to reconstruct low-resolution images, and high-resolution images are obtained through multiple coding sampling reconstructions; and a collaborative energy compensation algorithm is used for post-processing to remove the linear inconsistency effect that is easy to occur in optical coding imaging of linear array detectors.

[0057] Optical dynamic encoding is used to subdivide pixels to achieve sub-pixel sampling. The layout of the encoding board in the imaging system is as follows: Figure 2 As shown in the figure, the sub-pixel dynamic encoding plate is integrated closely to the surface of the TDI CMOS device. The 7um single pixel corresponds to a 2×2 3.5um sub-pixel optical encoding plate. The same object is encoded in 32 levels, and the signal is accumulated to output the linear array image of the first encoding. Then, the next 32-level TDI CMOS area uses a different encoding form to collect the linear array image of the same object. And so on, 8 linear array images with different encodings are collected, and the 8 low-resolution encoded images are sparsely reconstructed to obtain a high-resolution image. It is expected that the image resolution will be increased to twice the original.

[0058] The optical dynamic encoding method of the linear array TDI CMOS detector is shown in Figure 3(a) and Figure 3(b). In the first 32-level encoding, the first two columns of sub-pixel encoding are the same as the encoding of the next two columns, and the same encoding can be performed on the same ground object. After integration and accumulation, the image corresponding to the two columns of sub-pixel encoding in the first 32-level encoding is perceived by one column of detectors; the first two columns of sub-pixel encoding in the first 32-level are completely different from the first two columns of sub-pixel encoding in the N-1th and Nth 32-levels, so there are N different encodings. Sub-pixel encoding sampling is performed on the same column of ground objects through push-scanning to achieve optical dynamic encoding.

[0059] The local compressed sensing encoding method is: divide the high-resolution coded mask image of N×B pixels into image blocks of size B×B. j (j=1,2,…,n,n=(N / B 2)) is the column vector form of the jth block. Assume that a detector pixel corresponds to a high-resolution scene image in an R×R region, and the measurement image block size is K× Then the compressed sampling of the block can be expressed as:

[0060] y j =Φ j x j (1)

[0061] Among them, y j For the corresponding x j The column vector of the measured image patch, Φ j K 2 ×B 2 The size of the encoding matrix is ​​related to the actual encoding mask. As shown in Figure 4(a), the 8×8 pixel encoding mask corresponds to a 4×4 pixel detector array, where each detector pixel measures the intensity of the light field after encoding a 2×2 pixel subarray. After focal plane optical encoding, the detector does not obtain a high-resolution image of the observed scene, but a resolution compression sampling, and the image needs to be reconstructed through a sparse optimization algorithm. The resolution of the image is not determined by the detector, but by the pixel size of the encoding plate. Since the pixel size of the encoding plate is smaller than the detector pixel, the resolution of the imaging system can be significantly improved. Figure 4(b) shows the Φ image obtained using the high-resolution encoding mask corresponding to Figure 4(a). j During the construction process, the empty positions in the matrix are filled with zeros.

[0062] By changing different coding masks, L measurements are performed, and L pairs of coding matrices and measurement matrices are obtained for the same scene. Assume that the high-resolution scene image X is of size N×B, and the measurement image Y collected by the detector is Size, the local compressed sensing reconstruction process is as follows:

[0063] Step 1: Transform the high-resolution scene image X and the encoded mask image The image is divided into B×B blocks, and the measured image is divided into K×K blocks. According to the above method, the compressed sampling model of the i-th measurement can be constructed.

[0064]

[0065] in, They represent the column vector, encoding matrix, and column vector of the high-resolution scene image block of the i-th j-th measured image block respectively;

[0066] Step 2: Place L Extend and splice along the column vector direction to construct L measurement column vectors Similarly, construct L scene images and the encoding matrix Get L times measurement compression sampling model

[0067]

[0068] Step 3: Divide different types of remote sensing images such as forests and cities into B×B image blocks, use the K-SVD method to construct a complete dictionary, and obtain the remote sensing dictionary set ψ;

[0069] Step 4: Use the remote sensing dictionary set ψ and the StOMP algorithm to solve equation (3), and then solve the estimated values ​​of all high-resolution scene image blocks to obtain the initial value of the high-resolution scene image

[0070] Energy compensation is performed on high-resolution scene images using the following processing method:

[0071] Assuming that each pixel in the high-resolution scene image block has the same compensated grayscale value, the high-resolution scene image X and the encoded mask image Divided into R×R size image blocks, this corresponds to a detector pixel value. Then the energy compensation value corresponding to the mth image block is

[0072]

[0073] Among them, av(·) is the averaging operator, P m (·) is the operator for taking the mth R×R image block, D(e m ) indicates constructing an R×R image block, and each pixel value in the image block is e m , represents the mth R×R image block of the encoded mask image during the i-th measurement, It represents the measurement value of the mth detector in the ith measurement. By solving it with the first-order partial derivative equal to zero, we get

[0074]

[0075] Use the following formula to perform energy compensation for each high-resolution scene image block:

[0076]

[0077] Among them, X L ′ represents the final reconstructed high-resolution scene image.

[0078] By making full use of the prior knowledge of the continuity of the reconstructed image, the linear inconsistency effect that is prone to occur in optically coded imaging of linear array detectors is eliminated.

[0079] Optical dynamic coding imaging simulation of linear array detectors Figure 5As shown in the figure, first, the high-resolution line array original image and the high-resolution line array optical coding image are dot-multiplied to obtain the high-resolution line array coded imaging image, and then a 2×2 downsampling operation is performed to obtain a low-resolution coded line array image. Then, a one-dimensional vibration model is set, and multi-line array low-resolution coded images are spliced ​​considering the influence of vibration according to the model. Super-resolution reconstruction is performed with reference to the high-resolution coding image, and finally energy compensation is performed to reduce the linear inconsistency effect.

[0080] The preliminary simulation extracts the actual optical encoding plate calibration data as the optical encoding image for simulation. Since the optical encoding and the detector are not strictly aligned, there is a gray encoding area, and there are geometric distortions and blurring distortions. The simulation has integrated a variety of actual error sources, and the simulation results are closer to the actual imaging situation.

[0081] Using the target images collected in the laboratory for simulation analysis, it can be found that only two black ring targets can be seen in the low-resolution image, while six black ring targets can be distinguished in the high-resolution image. Starting from the reconstructed image obtained by using the optical encoding sampling image four times, six black ring targets can be distinguished, and the image quality also improves with the increase in the number of sampling times.

[0082] The present embodiment also provides a critical diffraction limit ultra-wide width high-resolution optical remote sensing imaging device, which includes: a first module, used to set a sub-pixel dynamic coding plate on the surface of a TDI CMOS device; a second module, used to sample a low-resolution image through the TDICMOS device to obtain a frame of coded image; a third module, used to replace different sub-pixel dynamic coding plates to obtain multiple frames of coded images; and a fourth module, used to reconstruct multiple frames of coded images to obtain a high-resolution image.

[0083] An electronic device comprises: a memory for storing computer-readable instructions; and a processor for running the computer-readable instructions to execute a critical diffraction limit ultra-wide width high-resolution optical remote sensing imaging method.

[0084] This embodiment only needs to cut in (cut out) the sub-pixel dynamic encoding plate in front of the focal plane to achieve an improvement in resolution, breaking through the technical bottleneck of improving the resolution of ultra-wide-width optical remote sensing cameras. It has low engineering implementation difficulty and strong scalability.

[0085] Although the present invention has been disclosed as above in the form of a preferred embodiment, it is not intended to limit the present invention. Any person skilled in the art may make possible changes and modifications to the technical solution of the present invention by using the methods and technical contents disclosed above without departing from the spirit and scope of the present invention. Therefore, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention shall fall within the protection scope of the technical solution of the present invention.

Claims

1. A critical diffraction-limited ultra-wideband high-resolution optical remote sensing imaging method, characterized in that include: Step S1: placing a sub-pixel dynamic coding plate on the surface of a TDI CMOS device; Step S2: sampling the low-resolution image through the TDICMOS device to obtain a frame of coded image; Step S3: Replace a different sub-pixel dynamic coding board, repeat steps S1 to S2, and obtain multiple frames of coded images; Step S4: reconstruct multiple frames of coded images to obtain high-resolution images.

2. The critical diffraction limit ultra-wide width high resolution optical remote sensing imaging method according to claim 1, characterized in that: The pixel size of the TDI CMOS device is 7um, and the pixel size of the sub-pixel dynamic encoding board is 3.5um.

3. The critical diffraction limit ultra-wide width high resolution optical remote sensing imaging method according to claim 2, characterized in that: Each pixel of the TDI CMOS device corresponds to a pixel of a 2×2 sub-pixel dynamic encoding panel.

4. The critical diffraction limit ultra-wide width high resolution optical remote sensing imaging method according to claim 2, characterized in that: In the sub-pixel dynamic coding board, the sub-pixel coding of the first two columns is the same as the sub-pixel coding of the adjacent two subsequent columns.

5. The critical diffraction limit ultra-wide width high resolution optical remote sensing imaging method according to claim 1 or 4, characterized in that: The encoding method of each sub-pixel dynamic encoding panel includes: Randomly generate W groups of encoding matrices, where W is a positive integer; Solve the condition number of the encoding matrix; Sort the condition numbers of the coding matrix from small to large to obtain the sequence number; The coding matrix corresponding to each serial number is cross-correlated and calculated to obtain the coding matrix with the minimum cross-correlation equal to the number of sub-pixel dynamic coding plates, and each coding matrix with the minimum cross-correlation is assigned to the corresponding sub-pixel dynamic coding plate.

6. The critical diffraction limit ultra-wide width high resolution optical remote sensing imaging method according to claim 5, characterized in that: W≥1000。 7. The critical diffraction limit ultra-wide width high resolution optical remote sensing imaging method according to claim 5, characterized in that: The encoding matrix has values ​​of [0,1].

8. The critical diffraction limit ultra-wide width high resolution optical remote sensing imaging method according to claim 5, characterized in that: The condition number of the encoding matrix is ​​obtained by the following formula: ∑1=diag(σ1,σ2,…,σ r ); Among them, C i is the condition number of the i-th group of encoding matrices, A i is the i-th group encoding matrix, min(·) represents the minimum value operator, max(·) represents the maximum value operator, is the encoding matrix A i The diagonal matrix after singular value decomposition, ∑1 is the non-negative real number on the diagonal, σ1 is the encoding matrix A i The first singular value of , σ2 is the encoding matrix A i The second singular value of r is the encoding matrix A i The rth singular value of i The number of non-zero singular values.

9. A critical diffraction limit ultra-wide width high resolution optical remote sensing imaging device, characterized in that include: The first module is used to set the sub-pixel dynamic coding plate on the surface of the TDICMOS device; The second module is used for sampling the low-resolution image through the TDICMOS device to obtain a frame of coded image; The third module is used to replace different sub-pixel dynamic coding plates to obtain multi-frame coded images; The fourth module is used to reconstruct multiple frames of coded images to obtain high-resolution images.

10. An electronic device, characterized in that: include: Memory: used to store computer-readable instructions; and Processor: used to run the computer-readable instructions to perform the method according to any one of claims 1 to 8.

Images