Multi-mode sparse aperture frequency division synthesis imaging method

Abstract

The invention relates to the technical field of space imaging, in particular to a multi-mode sparse aperture frequency division synthesis imaging method, which comprises the following steps of: performing rotary shooting on an observation target by adopting an imaging system integrated with a single aperture imaging subsystem and a sparse aperture imaging subsystem to obtain a low and medium frequency information image with the observation target; high-frequency information images corresponding to different rotation angles are obtained; constructing a loss function for judging an image fusion degree according to the low and medium frequency information image and the high frequency information image; and fusing the high-frequency information image and the low-and-medium-frequency information image, and carrying out iterative updating on the fused image based on a loss function to obtain a fused image. According to the invention, the single aperture imaging subsystem and the sparse aperture imaging subsystem are in rotation cooperation to respectively collect low and medium frequency information of a target area and high frequency information in multiple directions, the limitation of the carrying capacity of carrying equipment on the external size of the aperture is broken through, and the lightweight degree is improved at the same time.

Description

Technical Field

[0001] The present invention belongs to the technical field of space imaging, and particularly relates to a multi-mode sparse aperture frequency division synthesis imaging method. Background Art

[0002] In order to observe targets with a farther distance and lower brightness, the aperture of astronomical telescopes has been continuously increased. Space telescopes are located outside the Earth's atmosphere, and their resolution and imaging quality are superior to those of ground-based telescopes. However, due to the limitations of launch capacity, technical level, and research and development costs, the aperture of the main mirror of a single-body primary mirror imaging system generally does not exceed 4m. To break through this limitation, researchers have proposed various synthetic aperture imaging systems. Sparse synthetic aperture is similar to a segmented mirror. By analyzing the influence of the aperture on the transfer ability of spatial frequency information, the filling rate of the segmented synthetic aperture is reduced to achieve a lightweight effect.

[0003] In particular, there is a frequency division synthesis sparse aperture imaging method, which is a method of fusing the images collected by a sparse aperture for collecting high-frequency information and a single-aperture imaging subsystem for collecting medium and low-frequency information, further improving the lightweight degree of the sparse aperture. Specifically, this frequency division synthesis sparse aperture imaging method uses a single-body primary mirror to obtain the medium and low-frequency information of the target area, uses a sparse aperture imaging subsystem with a circular distribution to obtain the high-frequency information of the target area, and then fuses the high-frequency information and the medium and low-frequency information. The obtained fused image is the final imaging result. However, this imaging method has the technical problem that it is difficult to achieve an equivalent aperture of more than 4m due to the limitation of the outer contour size of the circularly distributed sparse aperture by the launch capacity.

[0004] To solve the problems caused by the size, there is also a rotating synthetic aperture imaging system in the existing methods, that is, the rotating synthetic aperture imaging system is set as a large aspect ratio rectangle, which can be vertically placed in the rocket launch section, and the long side breaks through the aperture limitation. However, this imaging system has the technical problems of poor mechanical properties of the large aspect ratio rectangular primary mirror, requiring complex manufacturing, surface shape control, system integration, and mirror body rotation technologies, as well as the technical problem of limited lightweight degree.

[0005] In addition, whether it is a rotating synthetic aperture imaging system or a frequency division synthesis sparse aperture imaging method, both use the method of directly adding pixels for image fusion. However, the method of directly adding pixels cannot accurately extract image information, resulting in a decrease in image contrast and a high degree of detail loss, ultimately leading to poor imaging effects and being unable to be used for subsequent applications such as image acquisition and processing. Summary of the Invention

[0006] In view of this, the present invention aims to provide a multi-mode sparse aperture frequency division synthesis imaging method, which rotates and cooperates a single-aperture imaging subsystem and a sparse aperture imaging subsystem to respectively collect medium and low frequency information and high frequency information in multiple directions of a target area, fuse the captured images, obtain an equivalent large-aperture high-resolution image, break through the limitation of the carrying capacity of the carrying device on the outer size of the aperture, obtain a larger equivalent aperture imaging effect, and at the same time improve the degree of lightweight.

[0007] To achieve the above object, the technical solution of the present invention is realized as follows: A multi-mode sparse aperture frequency division synthesis imaging method, comprising: S1: Using an imaging system integrated with a single-aperture imaging subsystem and a sparse aperture imaging subsystem to rotate and photograph an observation target, obtaining an image with medium and low frequency information of the observation target, and high frequency information images corresponding to different rotation angles; S2: According to the medium and low frequency information image and the high frequency information image obtained in step S1, constructing a loss function for judging the image fusion degree; S3: Fusing the high frequency information image obtained in step S1 with the medium and low frequency information image, and based on the loss function obtained in step S2, iteratively updating the fused image to obtain a fused image.

[0008] Further, in step S1: In each photograph taken by the single-aperture imaging subsystem, the medium and low frequency information of the observation target is included in the obtained medium and low frequency information image; on the sparse aperture imaging subsystem, there are at least 2 circular apertures arranged in columns. When the sparse aperture imaging subsystem rotates and photographs each time, the obtained high frequency information image has the high frequency information of the observation target along the direction of the center connection line of the circular apertures.

[0009] Further, in step S2, the loss function is: ; where E represents the loss function, N represents the total number of the medium and low frequency information image and the high frequency information image, n represents the number of images. When n = 0, n represents the medium and low frequency information image. When n ≥ 1, n represents the nth high frequency information image, I θn represents the frequency spectrum diagram corresponding to the high frequency information image captured when the imaging system is at the photographing angle θ n , I θ0 represents the frequency spectrum diagram corresponding to the medium and low frequency information image, OTF θn represents the frequency domain function of the point spread function of the imaging system at the photographing angle θ n , I m represents the frequency spectrum diagram of the fused image at the mth iterative update, 1 ≤ m ≤ M, and M represents the total number of iterations.

[0010] Further, the frequency-domain function OTF θn is obtained by the following formula: ; where P θn (x, y) represents the pupil function of the imaging system at the shooting angle θ n when, and represents the autocorrelation operation.

[0011] Further, in step S3, iteration is performed by the following formula: ; where α m,n represents the iteration factor for the nth image in the mth iteration, and λ m,n represents the regularization factor for the nth image in the mth iteration; After M iterations are completed, a frequency-domain to spatial-domain transformation is performed on the spectrogram updated in the Mth iteration to obtain and output the fused image.

[0012] Compared with the prior art, the present invention can achieve the following beneficial effects: (1) The multi-mode sparse aperture frequency division synthesis imaging method described in the present invention can realize that an imaging system integrated by a single aperture imaging subsystem and a sparse aperture imaging subsystem is vertically placed in a rocket carrier section, breaking through the limitation of the carrying capacity on the outer size of the aperture and obtaining a larger equivalent aperture imaging effect; in addition, compared with the rotating synthetic aperture imaging system, the sparse aperture imaging subsystem in the present invention that can obtain high-frequency information in a single direction is lighter in weight, has better mechanical properties for a single small circular mirror, and has more mature development technology; (2) The multi-mode sparse aperture frequency division synthesis imaging method described in the present invention adopts the Fourier spectrum restoration method, that is, based on the mid- and low-frequency information image collected by the single aperture imaging subsystem, its spectrum is obtained through Fourier transform, and then through iterative calculation, the high-frequency information images of the sparse aperture imaging subsystems at different angles are filled into the fused image. Compared with the existing fusion method of directly adding pixels, it can accurately extract image information, retain image contrast and image details, and has a good imaging effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The accompanying drawings forming a part of the present invention are used to provide a further understanding of the present invention. The schematic embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation to the present invention. In the drawings: Figure 1 is a schematic flowchart of the multi-mode sparse aperture frequency division synthesis imaging method described in the embodiment of the present invention; Figure 2Schematic diagram of the implementation process of the multi-mode sparse aperture frequency division synthesis imaging method described in the embodiments of the present invention; Figure 3 Schematic diagram of the pupil shape and frequency information transfer characteristics described in the embodiments of the present invention.

[0014] Explanation of reference numerals: 1. Single-aperture imaging subsystem; 2. Sparse-aperture imaging subsystem; 3. Medium- and low-frequency information image; 4. High-frequency information image; 5. Circular aperture; 6. Observation target; 7. Fusion image. Detailed implementation manners

[0015] In order to make the objectives, technical solutions and advantages of the present invention clearer and more understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and do not constitute a limitation to the present invention.

[0016] It should be noted that, without conflict, the embodiments in the present invention and the features in the embodiments may be combined with each other.

[0017] In the description of the present invention, it should be understood that the orientation or positional relationships indicated by the terms "center", "longitudinal", "transverse", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc. are based on the orientation or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as a limitation to the present invention. In addition, the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, the features defined with "first", "second", etc. may explicitly or implicitly include one or more of such features. In the description of the present invention, unless otherwise specified, the meaning of "plurality" is two or more.

[0018] In the description of the present invention, it should be noted that, unless otherwise clearly defined and limited, the terms "mounted", "connected" and "connected" should be understood in a broad sense. For example, it may be a fixed connection, a detachable connection or an integral connection; it may be a mechanical connection or an electrical connection; it may be directly connected or indirectly connected through an intermediate medium, and it may be the communication inside two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood through specific situations.

[0019] The present invention will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0020] The transfer ability of an imaging system to spatial frequencies is usually measured by the modulation transfer function MTF: ; where MTF(f x , f y ) is the modulus of the autocorrelation of the pupil function of the imaging system, f represents the spatial frequency in the x direction, f x represents the spatial frequency in the y direction, y represents the autocorrelation operation. For an imaging system composed of circular sub-apertures, the detailed expression of MTF is: For an imaging system composed of circular sub-apertures, the detailed expression of MTF is: ; where MTF s represents the modulation transfer function of a single aperture in the imaging system, λ refers to the working wavelength, f refers to the focal length of the imaging system, n refers to the total number of apertures in the imaging system, (x i -x j , y i -y j ) is the relative position of each aperture, and δ(·) represents the Dirac function, which is used to describe the relationship between the transfer function characteristics of the imaging system and the relative position of discrete sub-apertures. From the above formula, it can be obtained that: for a single circular aperture imaging system, the larger its aperture, the larger the range of spatial frequencies it can transfer (i.e., the larger the bandwidth); for a sparse aperture imaging system composed of multiple circular apertures, it can transfer information at specific spatial frequencies. This specific frequency is determined by the relative position (x i -x j , y i -y j ) between the apertures. The farther the relative position between the apertures, the higher the specific spatial frequency it can transfer (the corresponding value is higher). Based on the above analysis, the present invention provides a multi-mode sparse aperture frequency division synthesis imaging method in which a single aperture imaging subsystem and a sparse aperture imaging subsystem cooperate with each other.

[0021] As Figures 1 to 2 shown, the multi-mode sparse aperture frequency division synthesis imaging method described in the embodiment of the present invention includes: S1: Use an imaging system integrated with a single aperture imaging subsystem 1 and a sparse aperture imaging subsystem 2 to rotate and photograph an observation target 6, and obtain an image 3 with medium and low frequency information of the observation target 6, and high frequency information images 4 corresponding to different rotation angles.

[0022] In some embodiments, in each shot, the single-aperture imaging subsystem 1 captures the medium- and low-frequency information image 3 containing the medium- and low-frequency information of the observation target 6. The sparse-aperture imaging subsystem 2 is provided with no less than two circular apertures 5 arranged in columns. In each rotational shot, the high-frequency information image 4 captured by the sparse-aperture imaging subsystem 2 contains the high-frequency information of the observation target 6 along the direction connecting the centers of the circular apertures 5.

[0023] In the embodiments of the present invention, the single-aperture imaging subsystem 1 is a common imaging system on the market, and the single-aperture imaging subsystem 1 does not rotate. That is, during the rotation of the imaging system, the single-aperture imaging subsystem 1 does not rotate, while the sparse-aperture imaging subsystem 2 rotates stably at a fixed angle at a constant speed. The sparse-aperture imaging subsystem 2 provided by the embodiments of the present invention is based on the one-dimensional rotational synthetic aperture array disclosed in the paper "Imaging Characteristics of Sparse Aperture One-Dimensional Rotational Synthetic Array" in the journal Acta Optica Sinica, and is provided with two circular apertures 5 arranged in columns, and the aperture diameters of the single-aperture imaging subsystem 1 and the sparse-aperture imaging subsystem 2 satisfy the condition: the baseline length of the two circular apertures 5 (i.e., the distance between the centers of the two circular apertures 5) > the aperture diameter of the single-aperture imaging subsystem 1 > the aperture diameter of a single circular aperture 5. Compared with the existing one-dimensional rotational synthetic aperture array, the sparse-aperture imaging subsystem 2 provided by the embodiments of the present invention is more lightweight, and due to the addition of the single-aperture imaging subsystem 1, the single-aperture imaging subsystem 1 supplements the low-frequency information, and the rotational acquisition times of the sparse-aperture imaging subsystem 2 can be reduced.

[0024] The imaging process of the imaging system for the observation target 6 with rich texture information can be expressed by the following formula:

[0025] where i(x, y) represents the light intensity distribution of the image captured by the imaging system, that is, the captured medium- and low-frequency information image or high-frequency information image, o(x, y) represents the light intensity distribution function of the two-dimensional observation target, and PSF(x, y) represents the point spread function of the imaging system. The point spread function PSF is obtained by the following formula: ; ; where OTF represents the frequency domain function of the point spread function, O(u, v) represents the frequency domain distribution function of the light intensity distribution function o(x, y) after Fourier transform, I(u, v) represents the frequency domain distribution function of the light intensity distribution i(x, y) after Fourier transform, and P(x, y) represents the pupil function of the imaging system.

[0026] It can be seen from the imaging process of the imaging system for the observation target 6 that the pupil shape determines the information transfer ability of the imaging system, which in the frequency domain is manifested as transferring information of different frequencies.Figure 3 Shows the characteristics of frequency information transmission with different aperture diameters. Figure 3 The rectangle in [reference] is the spectrogram corresponding to the image obtained by the imaging system. Since the aperture diameters of the single-aperture imaging subsystem 1 and the sparse-aperture imaging subsystem 2 satisfy the above conditions, the single-aperture imaging subsystem 1 can transmit more medium and low-frequency information (dashed circle), and the sparse-aperture imaging subsystem 2 can collect a small amount of low-frequency information ( Figure 3 the middle solid circle among the three vertical solid circles in [reference]) and part of the high-frequency information in the arrangement direction of the circular aperture 5 ( Figure 3 the upper and lower solid circles among the three vertical solid circles in [reference]).

[0027] For the single-aperture imaging subsystem 1, its maximum aperture diameter determines the cut-off frequency (i.e., the highest frequency information that can be transmitted), and the frequency information transmitted by the single-aperture imaging subsystem 1 is Figure 3 the area of the dashed circle in [reference]. Because its aperture is larger than one of the sub-apertures, the range of the dashed circle is relatively large.

[0028] For the sparse-aperture imaging subsystem 2, the information it can transmit consists of two parts: the medium and low-frequency information corresponding to a single circular aperture 5, and the high-frequency information collected by two circular apertures 5 after the sparse-aperture imaging subsystem 2 rotates. For the medium and low-frequency information, since the aperture diameter of the circular aperture 5 of the sparse-aperture imaging subsystem 2 is smaller than that of the single-aperture imaging subsystem 1, the transmission ability of this part of the low-frequency information is significantly weaker than that of the single-aperture imaging subsystem 1, corresponding to Figure 3 the range of the solid circle in the sparse-aperture imaging subsystem 2 being smaller than that of the dashed circle. For the high-frequency information, the reason why the sparse-aperture imaging subsystem 2 can transmit high-frequency information is that there is a distance between the centers of the two circular apertures 5, and the distance is larger than the aperture diameter of the single-aperture imaging subsystem 1. Therefore, it can collect a small range of high-frequency information in the direction of the line connecting the centers of the two circular apertures 5. It can be understood that the distance between the centers of the two circular apertures 5 of the sparse-aperture imaging subsystem 2 determines the center positions of the two circular apertures 5 (i.e., the highest frequency that can be collected), and the size of the circular aperture 5 determines the bandwidth of the collection (i.e., how much frequency information near this high frequency can be transmitted). Therefore, the frequency information that the sparse-aperture imaging subsystem 2 can transmit is Figure 3 the range of the three solid circles in [reference].

[0029] The information synthesis after the sparse-aperture imaging subsystem 2 rotates is equivalent to a large ring. At this time, the single-aperture imaging subsystem 1 does not need to rotate to collect the frequency information in the middle dashed circle area. Integrating the high-frequency information and the medium and low-frequency information is equivalent to transmitting more frequency information.

[0030] It can be understood that each time the sparse aperture imaging subsystem 2 is controlled to perform rotational shooting, the high-frequency information in the high-frequency information image 4 corresponding to different rotational angles is different. Therefore, after controlling the sparse aperture imaging subsystem 2 to perform multiple rotational shootings on the observation target 6, all the high-frequency information of the observation target 6 can be obtained. The essence of the present invention is to obtain different spatial frequency information of the observation target through different imaging subsystems, and to obtain a high-resolution image of the omnidirectional high-frequency information of the observation target 6 through image fusion.

[0031] In the present invention, the image fusion process adopts the Fourier spectrum restoration method. Based on the medium and low-frequency information image 3 collected by the single aperture imaging subsystem 1, its spectrum is obtained through Fourier transform, and then through iterative calculation, the high-frequency information images 4 of the sparse aperture imaging subsystem 2 at different angles are filled into the fusion image 7. Specifically, it includes: S2: According to the medium and low-frequency information image 3 and the high-frequency information image 4 obtained in step S1, a loss function for judging the degree of image fusion is constructed.

[0032] In some embodiments, the loss function is: ; where E represents the loss function, N represents the total number of medium and low-frequency information images and high-frequency information images, n represents the number of images. When n = 0, n represents the medium and low-frequency information image, and when n ≥ 1, n represents the nth high-frequency information image. I θn represents the spectrogram corresponding to the high-frequency information image 4 captured when the imaging system is at the shooting angle θ n , I θ0 represents the spectrogram corresponding to the medium and low-frequency information image, OTF θn represents the frequency domain function of the point spread function of the imaging system at the shooting angle θ n , I m represents the spectrogram of the fusion image 7 at the mth iterative update, 1 ≤ m ≤ M, and M represents the total number of iterations. When the image fusion quality is good, the loss function E should approach 0.

[0033] In the embodiments of the present invention, the high-frequency information image 4 of the sparse aperture imaging subsystem 2 is subjected to Fourier transform to obtain the corresponding spectrogram I θn , that is: ; where F represents Fourier transform, i θn (x, y) represents the high-frequency information image 4 captured when the sparse aperture imaging subsystem 2 is at the shooting angle θ n ; The medium and low-frequency information image 3 captured by the single aperture imaging subsystem 1 is subjected to Fourier transform to obtain the corresponding spectrogram I θ0,Right now: ; Among them, i θ0 (x, y) represents the medium and low frequency information image 3.

[0034] Frequency domain function OTF θn Obtained by the following formula: ; Among them, P θn (x, y) indicates that the imaging system is at a shooting angle θ n The pupil function at Represents the autocorrelation operation.

[0035] S3: Fuse the high-frequency information image obtained in step S1 with the medium- and low-frequency information image, and iteratively update the fused image based on the loss function obtained in step S2 to obtain a fused image 7.

[0036] In some embodiments, iteration is performed by: ; Among them, α m,n represents the iteration factor of the nth image in the mth iteration, which is used to scale the iteration step and changes with the number of iterations and the difference between the iteration image and the medium-low frequency information image 3 and the high frequency information image 4; λ m,n It represents the regularization factor for the nth image in the mth iteration, which is used to ensure the correctness of the image iteration calculation and can be constructed by various methods such as image gradient and Laplace operator. After completing M iterations, the frequency domain to space domain transformation is performed on the spectrum map updated in the Mth iteration to obtain and output the fused image 7.

[0037] In the embodiment of the present invention, the regularization factor λ is constructed using the Laplace operator. m,n , specifically, the Laplace operator l is: ; Introducing the Laplace operator l into the loss function E yields: ; Among them, L represents the operator obtained by Fourier transform of Laplace operator l, and ρ represents the influence weight, which is used to adjust iteration and regularization. The value is adaptively adjusted according to actual conditions and experimental experience.

[0038] The corresponding iteration formula is: ; Among them, OTF θn * Represents frequency domain function OTF θn The complex conjugate of .

[0039] In an embodiment of the present invention, since the spectrogram is obtained by performing a Fourier transform on an image, the spectrogram obtained by the M-th iterative update is directly subjected to an inverse Fourier transform to obtain and output the fused image 7.

[0040] It should be understood that various forms of the processes shown above can be used, steps can be reordered, added or deleted. For example, the steps described in the disclosure of the present invention can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution disclosed in the present invention can be achieved, and no limitations are imposed herein.

[0041] The above specific embodiments do not constitute a limitation on the protection scope of the present invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims

1. A multi-mode sparse aperture frequency division synthesis imaging method, characterized in that: include: S1: Using an imaging system that integrates a single-aperture imaging subsystem and a sparse-aperture imaging subsystem to rotate and photograph an observation target, and obtain medium- and low-frequency information images of the observation target, as well as high-frequency information images corresponding to different rotation angles; S2: constructing a loss function for judging the degree of image fusion according to the medium and low frequency information image and the high frequency information image obtained in step S1; S3: Fuse the high-frequency information image obtained in step S1 with the medium- and low-frequency information image, and iteratively update the fused image based on the loss function obtained in step S2 to obtain a fused image.

2. The multi-mode sparse aperture frequency synthesis imaging method according to claim 1, characterized in that: In step S1: The single-aperture imaging subsystem captures medium- and low-frequency information images each time it takes pictures, which contain medium- and low-frequency information of the observed target; No less than two circular apertures are arranged in a row on the sparse aperture imaging subsystem. Each time the sparse aperture imaging subsystem rotates and shoots, the high-frequency information image captured has high-frequency information in the observed target along the direction of the line connecting the centers of the circular apertures.

3. The multi-mode sparse aperture frequency synthesis imaging method according to claim 1, characterized in that: In step S2, the loss function is: ； Wherein, E represents the loss function, N represents the total number of the medium-low frequency information images and the high frequency information images, n represents the number of images, when n=0, n represents the medium-low frequency information image, when n≥1, n represents the nth high frequency information image, I θn Indicates that the imaging system is at a shooting angle θ n The spectrum diagram corresponding to the high-frequency information image captured at θ0 The spectrum diagram corresponding to the medium and low frequency information image, OTF θn It represents the imaging system at the shooting angle θ n The frequency domain function of the point spread function when I m It represents the spectrum of the fused image at the mth iteration update, 1≤m≤M, and M represents the total number of iterations.

4. The multi-mode sparse aperture frequency synthesis imaging method according to claim 3, characterized in that: Frequency domain function OTF θn Obtained by the following formula: ； Among them, P θn (x, y) indicates that the imaging system is at a shooting angle θ n The pupil function at Represents the autocorrelation operation.

5. The multi-mode sparse aperture frequency synthesis imaging method according to claim 3, characterized in that: In step S3, the iteration is performed by the following formula: ； Among them, α m,n represents the iteration factor for the nth image in the mth iteration, λ m,n represents the regularization factor for the nth image in the mth iteration; After completing M iterations, the frequency domain to space domain transformation is performed on the spectrum graph updated in the Mth iteration to obtain and output the fused image.

Images