Multi-mode sparse aperture frequency division synthesis imaging method

Abstract

The present invention relates to the field of space imaging technology, and in particular to a multi-mode sparse aperture frequency division synthesis imaging method, comprising: using an imaging system that integrates a single aperture imaging subsystem and a sparse aperture imaging subsystem to rotate and shoot an observation target to obtain a medium- and low-frequency information image of the observation target, as well as high-frequency information images corresponding to different rotation angles; constructing a loss function for judging the degree of image fusion based on the medium- and low-frequency information images and the high-frequency information images; fusing the high-frequency information image with the medium- and low-frequency information image, and iteratively updating the fused image based on the loss function to obtain a fused image. The single aperture imaging subsystem and the sparse aperture imaging subsystem of the present invention rotate in coordination to respectively collect medium- and low-frequency information of the target area and high-frequency information in multiple directions, breaking through the limitation of the carrying capacity of the carrier equipment on the outer size of the aperture, while improving the degree of lightweighting.

Description

Technical Field

[0001] The present invention belongs to the technical field of space imaging, and in particular relates to a multi-mode sparse aperture frequency division synthesis imaging method. Background Art

[0002] To observe farther and fainter objects, the apertures of astronomical telescopes are constantly increasing. Space telescopes, located outside the Earth's atmosphere, offer superior resolution and imaging quality compared to ground-based telescopes. However, due to limitations in launch capacity, technological sophistication, and development costs, the primary mirror aperture of a single-lens primary mirror imaging system typically does not exceed 4 meters. To overcome this limitation, researchers have proposed various synthetic aperture imaging systems. Sparse synthetic apertures, similar to tiled mirrors, reduce the fill factor of the tiled synthetic aperture by analyzing the impact of aperture on the ability to transmit spatial frequency information, thereby achieving lightweighting.

[0003] In particular, a frequency-division synthesis sparse aperture imaging method fuses a sparse aperture that collects high-frequency information with images collected by a single aperture imaging subsystem for medium- and low-frequency information, further improving the lightweight nature of the sparse aperture. Specifically, this frequency-division synthesis sparse aperture imaging method uses a single primary mirror to obtain medium- and low-frequency information of the target area, and a ring-shaped sparse aperture imaging subsystem to obtain high-frequency information of the target area. The high-frequency information is then fused with the medium- and low-frequency information to create the final fused image. However, this imaging method has the technical problem that the outer contour size of the ring-shaped sparse aperture is still limited by the carrying capacity, making it difficult to achieve the equivalent of a large aperture of more than 4 meters.

[0004] To address the size issue, one existing approach is a rotating synthetic aperture imaging system (RSA). This system is designed as a large-aspect-ratio rectangular mirror, allowing it to be placed vertically in the launch vehicle compartment, with the long side exceeding the aperture limit. However, this imaging system suffers from the poor mechanical properties of the large-aspect-ratio rectangular primary mirror, requiring complex manufacturing, surface shape control, system integration, and mirror rotation technology, as well as limited lightweighting capabilities.

[0005] In addition, both the rotating synthetic aperture imaging system and the frequency-fractionation synthetic sparse aperture imaging method use the direct pixel addition method for image fusion. However, the direct pixel addition method cannot accurately extract image information, resulting in a decrease in image contrast and a high degree of detail loss, which ultimately leads to poor imaging effects and cannot be used for subsequent applications such as image acquisition and processing. Summary of the Invention

[0006] In view of this, the present invention aims to provide a multi-mode sparse aperture frequency division synthesis imaging method, which rotates and cooperates with a single aperture imaging subsystem and a sparse aperture imaging subsystem to respectively collect medium and low frequency information of the target area and high frequency information in multiple directions, fuse the captured images, and obtain equivalent large-aperture high-resolution images, breaking through the limitation of the carrying capacity of the carrier equipment on the outer size of the aperture, obtaining a larger equivalent aperture imaging effect, and at the same time improving the degree of lightweighting.

[0007] To achieve the above object, the technical solution created by the present invention is implemented as follows:

[0008] A multi-mode sparse aperture frequency division synthesis imaging method, comprising:

[0009] S1: An imaging system integrating a single-aperture imaging subsystem and a sparse-aperture imaging subsystem is used to rotate and photograph the observation target, obtaining images with medium- and low-frequency information of the observation target, as well as high-frequency information images corresponding to different rotation angles;

[0010] S2: Based on the low- and medium-frequency information images and the high-frequency information images obtained in step S1, a loss function is constructed to determine the degree of image fusion.

[0011] S3: Fuse the high-frequency information image obtained in step S1 with the medium- and low-frequency information image, and iteratively update the fused image based on the loss function obtained in step S2 to obtain a fused image.

[0012] Furthermore, in step S1: the single-aperture imaging subsystem captures medium- and low-frequency information images with medium- and low-frequency information of the observed target each time it takes a picture; and the sparse aperture imaging subsystem is provided with no less than two circular apertures in a column, and the sparse aperture imaging subsystem captures high-frequency information images with high-frequency information of the observed target along the direction of the line connecting the centers of the circular apertures each time it rotates and takes a picture.

[0013] Furthermore, in step S2, the loss function is:

[0014] ;

[0015] Among them, E represents the loss function, N represents the total number of images of medium-low-frequency information images and high-frequency information images, n represents the number of images, when n=0, n represents the medium-low-frequency information image, when n≥1, n represents the nth high-frequency information image, I θn Indicates that the imaging system is at a shooting angle θ n The spectrum diagram corresponding to the high-frequency information image captured at the time, I θ0 Indicates the spectrum corresponding to the low- and medium-frequency information image, OTF θn Indicates the imaging system at the shooting angle θ n The frequency domain function of the point spread function when Im It represents the spectrum of the fused image at the mth iteration update, 1≤m≤M, where M represents the total number of iterations.

[0016] Furthermore, the frequency domain function OTF θn It is obtained by the following formula:

[0017] ;

[0018] Among them, P θn (x,y) represents the imaging system at the shooting angle θ n The pupil function at Represents the autocorrelation operation.

[0019] Furthermore, in step S3, iteration is performed using the following formula:

[0020] ;

[0021] Among them, α m,n represents the iteration factor for the nth image in the mth iteration, λ m,n represents the regularization factor for the nth image in the mth iteration;

[0022] After completing M iterations, the spectrum graph obtained by the M-th iteration is transformed from the frequency domain to the spatial domain to obtain and output the fused image.

[0023] Compared with the prior art, the present invention can achieve the following beneficial effects:

[0024] (1) The multi-mode sparse aperture frequency division synthesis imaging method created by the present invention can realize the vertical placement of an imaging system integrated by a single aperture imaging subsystem and a sparse aperture imaging subsystem in a rocket carrier compartment, breaking through the limitation of the carrying capacity on the outer aperture size and obtaining a larger equivalent aperture imaging effect; in addition, compared with the rotating synthetic aperture imaging system, the sparse aperture imaging subsystem of the present invention that can obtain unidirectional high-frequency information is more lightweight, the mechanical properties of a single small circular reflector are better, and the development technology is more mature;

[0025] (2) The multi-mode sparse aperture frequency division synthesis imaging method created by the present invention adopts the Fourier spectrum restoration method, that is, based on the medium and low frequency information images collected by the single aperture imaging subsystem, its spectrum is obtained through Fourier transform, and then through iterative calculation, the high frequency information images of the sparse aperture imaging subsystem at different angles are filled into the fused image. Compared with the existing pixel direct addition fusion method, it can accurately realize image information extraction, retain image contrast and image details, and achieve good imaging effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The accompanying drawings, which constitute part of the present invention, are intended to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are intended to explain the present invention and do not constitute an undue limitation of the present invention. In the accompanying drawings:

[0027] Figure 1 A schematic diagram of the process flow of the multi-mode sparse aperture frequency division synthesis imaging method according to an embodiment of the present invention;

[0028] Figure 2 A schematic diagram illustrating the implementation process of the multi-mode sparse aperture frequency division synthesis imaging method described in an embodiment of the present invention;

[0029] Figure 3 Schematic diagram of pupil shape and frequency information transmission characteristics described in an embodiment of the present invention.

[0030] Description of reference numerals:

[0031] 1. Single aperture imaging subsystem; 2. Sparse aperture imaging subsystem; 3. Medium and low frequency information images; 4. High frequency information images; 5. Circular aperture; 6. Observation target; 7. Fusion image. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and do not constitute a limitation of the present invention.

[0033] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other.

[0034] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. In addition, the terms "first", "second" and the like are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, features defined as "first", "second" and the like may explicitly or implicitly include one or more of the features. In the description of the present invention, unless otherwise specified, "multiple" means two or more.

[0035] In the description of the present invention, it should be noted that, unless otherwise expressly specified or limited, the terms "installed," "connected," and "connected" should be understood in a broad sense. For example, they can refer to fixed connections, detachable connections, or integral connections; they can refer to mechanical connections or electrical connections; they can refer to direct connections or indirect connections through an intermediate medium; and they can refer to internal connections between two components. Those skilled in the art can understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0036] The present invention will be described in detail below with reference to the accompanying drawings and in combination with embodiments.

[0037] The imaging system's ability to transmit spatial frequencies is usually measured by the modulation transfer function (MTF):

[0038] ;

[0039] Among them, MTF (f x ,f y ) is the pupil function of the imaging system The modulus of the autocorrelation, f x represents the spatial frequency in the x direction, f y represents the spatial frequency in the y direction, Represents the autocorrelation operation. For an imaging system composed of circular sub-apertures, the detailed expression of MTF is:

[0040] ;

[0041] Among them, MTF s represents the modulation transfer function of a single aperture in the imaging system, λ refers to the operating wavelength, f refers to the focal length of the imaging system, and n refers to the total number of apertures in the imaging system. (x i -x j ,y i -y j ) is the relative position of each aperture, and δ(·) represents the Dirac function, which is used to describe the relationship between the transfer function characteristics of the imaging system and the relative positions of the discrete sub-apertures. From the above formula, it can be obtained that for a single circular aperture imaging system, the larger the aperture, the larger the spatial frequency range that can be transmitted (i.e., the larger the bandwidth); for a sparse aperture imaging system composed of multiple circular apertures, it can transmit information at a specific spatial frequency. This specific frequency is determined by the relative position between the sub-apertures (x i -x j ,y i -y j) is determined by the relative distance between the apertures. The greater the relative distance between the apertures, the higher the specific spatial frequency that can be transmitted (the corresponding numerical value is higher). Based on the above analysis, the present invention provides a multi-mode sparse aperture frequency division synthesis imaging method that cooperates with a single aperture imaging subsystem and a sparse aperture imaging subsystem.

[0042] like Figures 1 to 2 As shown, the multi-mode sparse aperture frequency synthesis imaging method described in the embodiment of the present invention includes:

[0043] S1: An imaging system integrating a single-aperture imaging subsystem 1 and a sparse-aperture imaging subsystem 2 is used to rotate and photograph an observation target 6, thereby obtaining a medium- and low-frequency information image 3 of the observation target 6, and a high-frequency information image 4 corresponding to different rotation angles.

[0044] In some embodiments, the single-aperture imaging subsystem 1 captures medium- and low-frequency information images 3 each time it takes a picture, which contain medium- and low-frequency information of the observation target 6; the sparse aperture imaging subsystem 2 is provided with no less than two circular apertures 5 in a column, and the sparse aperture imaging subsystem 2 captures high-frequency information images 4 each time it rotates and takes a picture, which contain high-frequency information of the observation target 6 along the direction of the line connecting the centers of the circular apertures 5.

[0045] In this embodiment of the present invention, the single-aperture imaging subsystem 1 is a common imaging system commonly found on the market, and the single-aperture imaging subsystem 1 does not rotate. That is, during the rotation of the imaging system, the single-aperture imaging subsystem 1 does not rotate, while the sparse-aperture imaging subsystem 2 rotates at a fixed angle and at a constant speed. The sparse-aperture imaging subsystem 2 provided in this embodiment of the present invention is based on the one-dimensional rotating synthetic aperture array disclosed in the paper "Imaging Characteristics of a Sparse Aperture One-Dimensional Rotating Synthetic Aperture Array" published in the journal Acta Optica Sinica. Two circular apertures 5 are arranged in a row, and the aperture diameters of the single-aperture imaging subsystem 1 and the sparse-aperture imaging subsystem 2 satisfy the following condition: the baseline length of the two circular apertures 5 (i.e., the distance between the centers of the two circular apertures 5) > the aperture diameter of the single-aperture imaging subsystem 1 > the aperture diameter of a single circular aperture 5. Compared with the existing one-dimensional rotating synthetic aperture array, the sparse aperture imaging subsystem 2 provided by the embodiment of the present invention is more lightweight, and due to the addition of the single aperture imaging subsystem 1, the single aperture imaging subsystem 1 supplements the low-frequency information, which can reduce the number of rotation acquisition times of the sparse aperture imaging subsystem 2.

[0046] The imaging process of the imaging system for the observation target 6 with rich texture information can be expressed by the following formula:

[0047]

[0048] Where i(x,y) represents the light intensity distribution of the image captured by the imaging system, that is, the captured medium- and low-frequency information image or high-frequency information image, o(x,y) represents the light intensity distribution function of the two-dimensional observation target, and PSF(x,y) represents the point spread function of the imaging system. The point spread function PSF is obtained by the following formula:

[0049] ;

[0050] ;

[0051] Among them, OTF represents the frequency domain function of the point spread function, O(u,v) represents the frequency domain distribution function of the light intensity distribution function o(x,y) after Fourier transform, I(u,v) represents the frequency domain distribution function of the light intensity distribution i(x,y) after Fourier transform, and P(x,y) represents the pupil function of the imaging system.

[0052] From the imaging process of the imaging system on the observation target 6, it can be seen that the pupil shape determines the information transmission capability of the imaging system, which is manifested in the frequency domain as the transmission of information of different frequencies. Figure 3 The frequency information transmission characteristics of different apertures are demonstrated. Figure 3 The rectangle in the figure is the spectrum diagram corresponding to the image acquired by the imaging system. Since the aperture diameters of the single-aperture imaging subsystem 1 and the sparse-aperture imaging subsystem 2 meet the above conditions, the single-aperture imaging subsystem 1 can transmit more medium- and low-frequency information (dashed circle), and the sparse-aperture imaging subsystem 2 can collect a small amount of low-frequency information ( Figure 3 The middle solid circle among the three solid circles in the middle column) and part of the high-frequency information in the arrangement direction of the circular aperture 5 ( Figure 3 The upper and lower two solid circles in the middle vertical column of three solid circles).

[0053] For the single-aperture imaging subsystem 1, its maximum aperture diameter determines the cutoff frequency (i.e., the maximum frequency information that can be transmitted). The frequency information transmitted by the single-aperture imaging subsystem 1 is Figure 3 The dotted circle area in . Because its aperture is larger than one of the sub-apertures in , the dotted circle has a larger range.

[0054] For the sparse aperture imaging subsystem 2, the information it can transmit consists of two parts: the medium and low frequency information corresponding to the single circular aperture 5, and the high frequency information collected by the two circular apertures 5 after the sparse aperture imaging subsystem 2 rotates. As for the medium and low frequency information, since the aperture diameter of the circular aperture 5 of the sparse aperture imaging subsystem 2 is smaller than that of the single aperture imaging subsystem 1, the transmission capability of this part of the low frequency information is significantly weaker than that of the single aperture imaging subsystem 1, corresponding to Figure 3The range of the solid circle in the sparse aperture imaging subsystem 2 is smaller than that of the dotted circle. As for high-frequency information, the sparse aperture imaging subsystem 2 can transmit high-frequency information because there is a distance between the centers of the two circular apertures 5, and the distance is larger than the aperture diameter of the single aperture imaging subsystem 1. Therefore, it can collect a small range of high-frequency information in the direction of the line connecting the centers of the two circular apertures 5. It can be understood that the distance between the centers of the two circular apertures 5 of the sparse aperture imaging subsystem 2 determines the position of the centers of the two circular apertures 5 (that is, how high a frequency can be collected), and the size of the circular aperture 5 determines the bandwidth of the collection (that is, how much frequency information near this high frequency can be transmitted). Therefore, the frequency information that the sparse aperture imaging subsystem 2 can transmit is Figure 3 The range of the three solid circles.

[0055] The information generated by the rotating sparse aperture imaging subsystem 2 forms a large ring. The single aperture imaging subsystem 1, without rotating, can now collect frequency information within the dotted circle. This integration of high-frequency information with mid- and low-frequency information effectively transmits more frequency information.

[0056] It will be appreciated that each time the sparse aperture imaging subsystem 2 is controlled to rotate and capture images, different rotation angles correspond to different high-frequency information in the high-frequency information image 4. Therefore, by controlling the sparse aperture imaging subsystem 2 to rotate and capture the observation target 6 multiple times, all high-frequency information of the observation target 6 can be obtained. The essence of the present invention is to obtain different spatial frequency information of the observation target using different imaging subsystems, and to obtain a high-resolution image of the omnidirectional high-frequency information of the observation target 6 through image fusion.

[0057] The image fusion processing in the present invention adopts the Fourier spectrum restoration method. Based on the medium and low frequency information image 3 collected by the single aperture imaging subsystem 1, its spectrum is obtained through Fourier transform. Then, through iterative calculation, the high frequency information image 4 of the sparse aperture imaging subsystem 2 at different angles is filled into the fused image 7. Specifically, it includes:

[0058] S2: Based on the low- and medium-frequency information image 3 and the high-frequency information image 4 obtained in step S1, a loss function is constructed to determine the degree of image fusion.

[0059] In some embodiments, the loss function is:

[0060] ;

[0061] Where E represents the loss function, N represents the total number of medium- and low-frequency information images and high-frequency information images, and n represents the number of images. When n=0, n represents the medium- and low-frequency information image, and when n≥1, n represents the nth high-frequency information image. θn Indicates that the imaging system is at a shooting angle θ nThe spectrum diagram corresponding to the high-frequency information image 4 captured at the time, I θ0 Indicates the spectrum corresponding to the low- and medium-frequency information image, OTF θn Indicates the imaging system at the shooting angle θ n The frequency domain function of the point spread function when I m The spectrum of the fused image 7 at the mth iteration update is shown, 1≤m≤M, and M represents the total number of iterations. When the image fusion quality is good, the loss function E should be close to 0.

[0062] In the embodiment of the present invention, the high-frequency information image 4 of the sparse aperture imaging subsystem 2 is Fourier transformed to obtain the corresponding spectrum image I θn ,Right now:

[0063] ;

[0064] Where F represents Fourier transform, i θn (x, y) represents the sparse aperture imaging subsystem 2 at the shooting angle θ n High-frequency information image 4 captured at the time;

[0065] Perform Fourier transform on the medium and low frequency information image 3 captured by the single aperture imaging subsystem 1 to obtain the corresponding spectrum graph I θ0 ,Right now:

[0066] ;

[0067] Among them, i θ0 (x,y) represents the medium and low frequency information image 3.

[0068] Frequency domain function (OTF) θn It is obtained by the following formula:

[0069] ;

[0070] Among them, P θn (x,y) represents the imaging system at the shooting angle θ n The pupil function at Represents the autocorrelation operation.

[0071] S3: Fuse the high-frequency information image obtained in step S1 with the medium- and low-frequency information image, and iteratively update the fused image based on the loss function obtained in step S2 to obtain a fused image 7.

[0072] In some embodiments, the iteration is performed by:

[0073] ;

[0074] Among them, α m,nIt represents the iteration factor of the nth image in the mth iteration, which is used to scale the iteration step size and changes with the number of iterations and the difference between the iterative image and the low- and medium-frequency information image 3 and the high-frequency information image 4; λ m,n Represents the regularization factor for the nth image in the mth iteration, used to ensure the correctness of the image iterative calculation. It can be constructed using various methods, such as image gradients and Laplacian operators. After completing M iterations, the frequency domain to spatial domain transformation is performed on the updated spectrogram from the Mth iteration to obtain and output the fused image 7.

[0075] In the embodiment of the present invention, the regularization factor λ is constructed using the Laplace operator. m,n , specifically, the Laplace operator l is:

[0076] ;

[0077] Introducing the Laplace operator l into the loss function E yields:

[0078] ;

[0079] Where L represents the operator obtained by Fourier transform of the Laplace operator l, and ρ represents the influence weight, which is used to adjust iteration and regularization. The value is adaptively adjusted according to actual conditions and experimental experience.

[0080] The corresponding iterative formula is:

[0081] ;

[0082] Among them, OTF θn * Represents frequency domain function OTF θn The complex conjugate of .

[0083] In the embodiment of the present invention, since the spectrum graph is obtained by performing Fourier transform on the image, the spectrum graph obtained by the M-th iterative update is directly subjected to inverse Fourier transform to obtain and output the fused image 7.

[0084] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in the present disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved. This is not limited herein.

[0085] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. A multi-mode sparse aperture frequency synthesis imaging method, characterized in that: include: S1: Using an imaging system that integrates a single-aperture imaging subsystem and a sparse-aperture imaging subsystem to rotate and photograph an observation target, obtaining images with medium- and low-frequency information of the observation target, as well as high-frequency information images corresponding to different rotation angles; S2: Based on the low- and medium-frequency information images and the high-frequency information images obtained in step S1, a loss function for judging the degree of image fusion is constructed. The loss function is: ； Wherein, E represents the loss function, N represents the total number of the medium-low frequency information images and the high frequency information images, n represents the number of images, when n=0, n represents the medium-low frequency information image, when n≥1, n represents the nth high frequency information image, I θn Indicates that the imaging system is at a shooting angle θ n The spectrum diagram corresponding to the high-frequency information image captured at the time, I θ0 Indicates the spectrum corresponding to the medium and low frequency information image, OTF θn Indicates the imaging system at the shooting angle θ n The frequency domain function of the point spread function when I m represents the spectrum of the fused image at the mth iteration update, 1≤m≤M, M represents the total number of iterations; S3: Fuse the high-frequency information image obtained in step S1 with the medium- and low-frequency information image, and iteratively update the fused image based on the loss function obtained in step S2 to obtain a fused image.

2. The multi-mode sparse aperture frequency synthesis imaging method according to claim 1, characterized in that: In step S1: The single-aperture imaging subsystem captures medium- and low-frequency information images each time, which contain medium- and low-frequency information of the observed target. No less than two circular apertures are arranged in a column on the sparse aperture imaging subsystem. When the sparse aperture imaging subsystem rotates and shoots each time, the high-frequency information image captured has high-frequency information in the observed target along the direction of the line connecting the centers of the circular apertures.

3. The multi-mode sparse aperture frequency synthesis imaging method according to claim 1, characterized in that: Frequency domain function (OTF) θn It is obtained by the following formula: ； Among them, P θn (x, y) represents the imaging system at a shooting angle θ n The pupil function at Represents the autocorrelation operation.

4. The multi-mode sparse aperture frequency synthesis imaging method according to claim 1, characterized in that: In step S3, the iteration is performed by the following formula: ； Among them, α m,n represents the iteration factor for the nth image in the mth iteration, λ m,n represents the regularization factor for the nth image in the mth iteration; After completing M iterations, the spectrum graph obtained by the M-th iteration is transformed from the frequency domain to the spatial domain to obtain and output the fused image.

Images