An optical synthetic aperture dynamic variable array imaging system and imaging method

Abstract

The present invention relates to an optical synthetic aperture dynamic variable array imaging system and an imaging method. The imaging system includes a multi-frame image acquisition module and an image restoration module. Among them, the multi-frame image acquisition module includes an optical synthetic aperture dynamic variable array and a detector connected in sequence. The optical synthetic aperture dynamic variable array includes a plurality of sub-apertures, and the plurality of sub-apertures form an aperture array with an X-shaped baseline. By using the dynamic variable array method of rotating the aperture array and stretching the baseline, multi-frame image acquisition of the object to be measured by the detector is realized. The image restoration module uses the trained deep learning model to restore the multi-frame images to obtain restored images. The imaging system of the present invention is provided with an optical synthetic aperture dynamic variable array, and the imaging effect of an equivalent large aperture is realized by using a small aperture sparse system through the variable array method of rotation and stretching. And the multi-frame observation image group generated by the system is restored by a deep learning algorithm to achieve high-resolution imaging.

Description

Technical Field

[0001] The present invention belongs to the field of optical synthetic aperture imaging, and particularly relates to an optical synthetic aperture dynamic variable array imaging system and an imaging method. Background Art

[0002] Optical remote sensing technology is developing towards high spatial, high spectral and high temporal resolutions. High spatial resolution imaging technology is widely applied in fields such as military reconnaissance, resource exploration, and astronomical observation. Due to the limitation of the diffraction limit, if the angular resolution of the system is to be improved, only the system aperture can be increased. The increase in the aperture will inevitably lead to an increase in the volume and mass of the system, bringing difficulties to the launch of space-based optical systems. Therefore, optical synthetic aperture emerges as the times require.

[0003] The optical synthetic aperture arranges several sub-apertures in a certain way. Under the condition of meeting the phase synchronization, interference imaging is realized on the focal plane, so as to achieve the resolution equivalent to that of a large-aperture telescope. However, since the light passing area of the optical synthetic aperture system is only partially filled with that of the equivalent single-aperture system, the response to the low and medium parts of the spatial frequency is reduced or aliased and suppressed, inevitably resulting in a decrease in image contrast and a phenomenon of blurring and degradation. Different sub-aperture arrangements have different imaging effects. Therefore, it is necessary to perform image restoration on the directly obtained image of the system to obtain a high-resolution remote sensing image.

[0004] At present, there are mainly two ways to improve the resolution. One is to optimize the aperture structure to improve the imaging effect, and the other is to improve the restoration algorithm. Most of the existing sub-aperture optimizations are optimizations or rearrangements of the annular or Golay type. The studied system apertures are all relatively small and the filling ratio is relatively high, which is difficult to meet the requirements of space observation. The existing restoration algorithms for optical synthetic aperture are all traditional algorithms. They are all based on a relatively simple and reasonable assumption. According to the image degradation model, a corresponding restoration model is established, and then iterative inversion is carried out. Most of the degradation models basically include the convolution term of the original image and the system transfer function of the degradation system. However, deconvolution is a ill-posed problem in theory and is prone to the situation of unstable solution. If the iterative process is regularized and constrained according to the expert prior knowledge, the complexity of the algorithm will be increased and the restoration efficiency will be reduced. In addition, there are also problems such as complex constraint conditions and relatively high algorithm complexity, which are difficult to realize the fast restoration work for various scenarios; moreover, there are few algorithms for multi-frame restoration, or they are computationally complex, or the prior conditions are harsh, lacking universality. Summary of the Invention

[0005] In order to solve the above problems existing in the prior art, the present invention provides an optical synthetic aperture dynamic variable array imaging system and an imaging method. The technical problems to be solved by the present invention are realized through the following technical solutions:

[0006] The present invention provides an optical synthetic aperture dynamic variable array imaging system, including: a multi-frame image acquisition module and an image restoration module, where,

[0007] The multi-frame image acquisition module includes an optical synthetic aperture dynamic variable array and a detector connected in sequence, where the optical synthetic aperture dynamic variable array includes a plurality of sub-apertures, and the plurality of sub-apertures form an aperture array with an X-shaped baseline. By using the dynamic variable array method of rotating the aperture array and stretching and contracting the baseline, the detector can collect multi-frame images of the object to be measured;

[0008] The image restoration module uses a trained deep learning model to restore the multi-frame images to obtain restored images.

[0009] In an embodiment of the present invention, the baseline is the connection line between the center of the circumscribed circle of the aperture array and the center of the sub-aperture.

[0010] In an embodiment of the present invention, the aperture array is a four-aperture array with an X-shaped baseline. During the dynamic variable array process, the diameter of the circumscribed circle of the four-aperture array does not exceed 1 m.

[0011] In an embodiment of the present invention, the diameter of the sub-aperture is 50-100 mm; the angular range of the X-shaped baseline formed by the aperture array is π / 8-π / 2; during the dynamic variable array process, the aperture array rotates within the angular range of 0 to 2π, and the rotation angle is π / 8-π / 2.

[0012] In an embodiment of the present invention, during the dynamic variable array process, the stretching and contracting step length of the baseline is 40 mm-70 mm.

[0013] In an embodiment of the present invention, the X-shaped baseline formed by the aperture array is asymmetric, where the difference between the longer baseline and the shorter baseline is equal to the diameter of the sub-aperture.

[0014] In an embodiment of the present invention, the deep learning model adopts a Uformer network structure, and the Uformer network structure includes: an input mapping module, a plurality of encoder modules, a plurality of decoder modules, and an output mapping module, where,

[0015] The input mapping module is used to perform primary feature extraction on the input multi-frame images to obtain a primary feature map;

[0016] The encoder module includes a cascaded LeWin Transformer block and a downsampling layer, and the encoder module is used to perform feature extraction and fusion on the primary feature map to obtain a fusion feature map;

[0017] The decoder module includes a cascaded upsampling layer and a LeWin Transformer block, and the decoder module is used to learn and recover the fused feature map to obtain a one-dimensional feature map;

[0018] The output mapping module is used to perform feature mapping on the one-dimensional feature map to obtain the restored image.

[0019] In an embodiment of the present invention, the LeWin Transformer block is a locally enhanced window transformer block, including a window-based multi-head self-attention module and a locally enhanced feed-forward network. Among them, the window-based multi-head self-attention module uses the self-attention mechanism to capture global dependencies, and the locally enhanced feed-forward network uses depth convolution to capture local information.

[0020] The present invention provides an optical synthetic aperture dynamic variable array imaging method, including:

[0021] Step 1: Use a four-aperture array with an X-shaped baseline as the initial array. Among them, the angle of the X-shaped baseline is π / 8, and the X-shaped baseline formed by the four-aperture array is asymmetric. The difference between the longer baseline and the shorter baseline is equal to the diameter of the sub-aperture;

[0022] Step 2: Change the baseline length with a telescopic step of 50 mm, and at the same time use a detector to collect images of the object to be measured corresponding to different baseline lengths. Among them, the diameter of the circumscribed circle of the four-aperture array does not exceed 1 m;

[0023] Step 3: Rotate the four-aperture array one week with an angle of π / 8, and repeat Step 2 after each rotation to obtain multiple frames of images of the object to be measured;

[0024] Step 4: Input the multiple frames of images into the trained deep learning model to obtain the restored image.

[0025] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0026] 1. The optical synthetic aperture dynamic variable array imaging system of the present invention is provided with an optical synthetic aperture dynamic variable array. Through the variable array method of rotation and telescoping, the imaging effect of an equivalent large aperture is achieved with a small aperture sparse system, and the multiple frames of observation image groups generated by the system are restored through a deep learning algorithm to achieve high-resolution imaging.

[0027] 2. The optical synthetic aperture dynamic variable array imaging method of the present invention restores the optical synthetic aperture image through deep learning, combines deep learning and the variable array system, and combines the multi-channel characteristics of deep learning and the multiple frames of images of the variable array system to break through the problem of difficult processing of multiple frames of images by traditional algorithms.

[0028] The above description is only an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention, it can be implemented according to the content of the specification. And in order to make the above and other purposes, features and advantages of the present invention more obvious and understandable, the following preferred embodiments are specifically given and described in detail in conjunction with the accompanying drawings. Description of the Drawings

[0029] Figure 1 It is a structural block diagram of an optical synthetic aperture dynamic variable array imaging system provided by an embodiment of the present invention;

[0030] Figure 2 It is a schematic diagram of the imaging principle of an optical synthetic aperture dynamic variable array provided by an embodiment of the present invention;

[0031] Figure 3 It is a schematic diagram of parameter optimization of an optical synthetic aperture dynamic variable array provided by an embodiment of the present invention;

[0032] Figure 4 It is a schematic diagram of the initial array, aperture arrangement and corresponding MTF diagram of an optical synthetic aperture dynamic variable array provided by an embodiment of the present invention;

[0033] Figure 5 It is a schematic diagram of a simulated multi-frame image provided by an embodiment of the present invention;

[0034] Figure 6 It is a schematic diagram of the comparison of MTF between various optical synthetic aperture dynamic variable arrays and equivalent apertures provided by an embodiment of the present invention;

[0035] Figure 7 It is a flowchart of the construction and training of a deep learning model provided by an embodiment of the present invention;

[0036] Figure 8 It is a schematic diagram of the Uformer network structure provided by an embodiment of the present invention;

[0037] Figure 9 It is a schematic diagram of the structure of a LeWin Transformer block provided by an embodiment of the present invention;

[0038] Figure 10 It is a comparison diagram of the restoration effects of different networks of Y-shaped and ring-shaped array systems provided by an embodiment of the present invention;

[0039] Figure 11 It is a comparison diagram of the visual effects of restored images of different imaging methods provided by an embodiment of the present invention;

[0040] Figure 12 It is a flowchart of an optical synthetic aperture dynamic variable array imaging method provided by an embodiment of the present invention. Detailed Embodiments

[0041] In order to further elaborate on the technical means and effects adopted by the present invention to achieve the intended invention purpose, the following will, in combination with the accompanying drawings and specific embodiments, provide a detailed description of an optical synthetic aperture dynamic variable array imaging system and an imaging method proposed according to the present invention.

[0042] Regarding the foregoing and other technical contents, features, and effects of the present invention, they can be clearly presented in the following detailed description in conjunction with the accompanying drawings. Through the description of the specific embodiments, a more in-depth and specific understanding of the technical means and effects adopted by the present invention to achieve the intended purpose can be obtained. However, the accompanying drawings are only for reference and explanation, and are not used to limit the technical solutions of the present invention.

[0043] Embodiment 1

[0044] Please refer to Figure 1 , Figure 1 which is a structural block diagram of an optical synthetic aperture dynamic variable array imaging system provided by an embodiment of the present invention. As shown in the figure, the optical synthetic aperture dynamic variable array imaging system of this embodiment includes a multi-frame image acquisition module and an image restoration module. Among them, the multi-frame image acquisition module includes an optical synthetic aperture dynamic variable array and a detector connected in sequence. Among them, the optical synthetic aperture dynamic variable array includes a plurality of sub-apertures, and the plurality of sub-apertures form an aperture array with an X-shaped baseline. By using the dynamic variable array method of rotating the aperture array and stretching and contracting the baseline, multi-frame image acquisition of the object to be measured by the detector is realized; the image restoration module uses the trained deep learning model to restore the multi-frame images to obtain restored images.

[0045] It should be noted that the baseline is the connection line between the center of the circumscribed circle of the aperture array and the center of the sub-aperture.

[0046] In this embodiment, the aperture array is a four-aperture array with an X-shaped baseline. During the dynamic variable array process, the diameter of the circumscribed circle of the four-aperture array does not exceed 1 m.

[0047] Optionally, the diameter of the sub-aperture is 50 - 100 mm; the angle of the X-shaped baseline formed by the aperture array (the angle α shown in Figure 2 ) ranges from π / 8 to π / 2; during the dynamic variable array process, the aperture array rotates within the angular range of 0 to 2π, and the rotation angle is π / 8 to π / 2, that is, the overall rotation angle range of the aperture array is 0 to 2π, and the rotation angle each time is based on the previous time and ranges from π / 8 to π / 2.

[0048] Optionally, during the dynamic variable array process, the stretching and contracting step length of the baseline is 40 mm - 70 mm.

[0049] Please refer to in combination Figure 2 , Figure 2It is a schematic diagram of the imaging principle of an optically synthetic aperture dynamic variable array provided by an embodiment of the present invention; as shown in the figure, the angle of the X-shaped baseline formed by the optically synthetic aperture dynamic variable array in this embodiment is variable, and it stays for a period of time after each rotation or telescopic movement, and a frame of image is observed. The spectrum covers a certain range. After obtaining multiple frames of images, the spectrum coverage is complete, and complete imaging information can be obtained by synthesizing the information between multiple frames of imaging.

[0050] Further, the X-shaped baseline formed by the aperture array is asymmetric, wherein the difference between the longer baseline and the shorter baseline is equal to the diameter of the sub-aperture.

[0051] In this embodiment, preferably, the diameter of the sub-aperture is 70 mm, the angle of the X-shaped baseline is π / 8, the telescopic step of the baseline is 50 mm. During the dynamic variable array process, the aperture array rotates one week (2π) with π / 8 as the rotation angle. In the asymmetric X-shaped baseline, the difference between the longer baseline and the shorter baseline is 70 mm.

[0052] Further, the optimization process of the structural parameters (the angle of the X-shaped baseline, the rotation angle of the aperture array, and the telescopic step) of the optically synthetic aperture dynamic variable array in this embodiment is described. In this embodiment, the minimum image reconstruction error criterion is used to optimize and determine these parameters. The minimum image reconstruction error criterion means that by comparing different variable array methods, multi-frame Wiener filtering is used for the images obtained by the variable array, and the correlation coefficient is used as the evaluation index. The largest correlation coefficient indicates the smallest reconstruction error, that is, the variable array method is the best when the correlation coefficient is the largest.

[0053] Please refer to Figure 3 , Figure 3 which is a schematic diagram of parameter optimization of an optically synthetic aperture dynamic variable array provided by an embodiment of the present invention, and specifically includes the following steps:

[0054] Step 1a, determining the initial array: Since the angle of the X-shaped baseline formed by the aperture array is variable, therefore, without considering noise and phase error, four-aperture arrays with the angles of the X-shaped baseline being π / 8, π / 6, π / 4, π / 3, and π / 2 are used as the initial arrays respectively, and they are rotated one week with the corresponding angles of the X-shaped baseline as the rotation angles of the aperture array, and multi-frame Wiener filtering restoration is performed. The variation relationship of the obtained correlation coefficient with the rotation angle is obtained, and it is found that the restoration effect is the best when the aperture array rotates one week with π / 8.

[0055] Step 1b, determining the rotation angle: Experiments were conducted on the same initial array as in Step 1a with a rotation angle of π / 8 for the aperture array. The relationship between the restoration correlation coefficient of different initial arrays and the rotation angle was obtained. It was found that the initial array with an X-shaped baseline angle of π / 8 had the best effect. Therefore, the initial array was determined to be a four-aperture array with an X-shaped baseline angle of π / 8. Furthermore, according to Step 1a, the rotation angle of the aperture array was also determined to be π / 8;

[0056] Step 1c, determining the configuration scaling step: Select the four-aperture array with an X-shaped baseline angle of π / 8 as the initial array for rotation and its rotation angle. To reduce redundancy, rotate a total of π, and conduct scaling experiments at different step intervals. The sub-aperture diameter is 70 mm. Considering that the baseline has a maximum scaling step of 70 mm, and to avoid excessive redundancy, the minimum step is set to 40 mm. Therefore, at intervals of 40 mm, 50 mm, 60 mm, and 70 mm, the relationship between the correlation coefficient and the baseline length at different scaling steps was obtained. It was found that when the scaling step was 50 mm, the restoration effect was the best.

[0057] Step 1d, configuration optimization: After the variable array method is initially determined, observe whether the spectral coverage map is complete. Please refer to Figure 4 , Figure 4 is a schematic diagram of the initial array, aperture arrangement, and the corresponding MTF diagram of an optical synthetic aperture dynamic variable array provided by an embodiment of the present invention. It can be seen from the figure that the above variable array method has incomplete spectral coverage and missing intermediate frequencies. After analysis, it is known that when the angle of the X-shaped baseline formed by the aperture array is π / 8, the baseline of the aperture cannot be shortened. To cover the intermediate frequencies, consider setting the X-shaped baseline to be asymmetric, that is, shortening one of the sides of the X-shaped baseline. The difference between the longer baseline and the shorter baseline is set to the diameter of the sub-aperture, 70 mm. Then, the baselines in the initial array are set to 100 mm and 170 mm, with a rotation angle of π / 8 and a scaling step of 50 mm. After scaling and rotation, a total of 56 frames of images are obtained, as shown in Figure 5 shown, Figure 5 is a schematic diagram of a simulation multi-frame image provided by an embodiment of the present invention.

[0058] Furthermore, to evaluate the effectiveness of the optical synthetic aperture dynamic variable array of this embodiment, a comparative experiment was conducted on the optical synthetic aperture dynamic variable array, symmetric variable array, and uniform initial array variable array with an X-shaped baseline angle of π / 2. Among them, the rotation angle of the aperture array and the maximum baseline scaling length were the same. The obtained MTF normalized frequency curve is as shown in Figure 6 shown, Figure 6It is a schematic diagram comparing the multiple optical synthetic aperture dynamic variable arrays and the equivalent aperture MTF provided by the embodiments of the present invention. As can be seen from the figure, when rotating symmetrically at 22.5°, the MTF is close to 0 between the intermediate frequencies of about 0.21 to 0.27. This is because the symmetric apertures expand and contract simultaneously, and when the baseline is 180 mm, the sub-apertures will be tangent to each other, so that the intermediate frequencies cannot be covered. When the initial array angle is π / 2, the intermediate frequencies can be covered, but the cut-off frequency can only reach about 0.9 at most, and the equivalent aperture is small. Under the same rotation angle and number of times, the asymmetric aperture can achieve a higher spectral coverage rate and the intermediate frequencies will not be missing, achieving the highest efficiency.

[0059] Further, the construction and training process of the deep learning model in this embodiment will be specifically described. Please refer to Figure 7 , Figure 7 It is a flowchart for constructing and training a deep learning model provided by the embodiments of the present invention. As shown in the figure, the construction and training process of the deep learning model in this embodiment includes the following steps:

[0060] Step 2a: Construct a deep learning network.

[0061] Specifically, in this embodiment, the deep learning model adopts the Uformer network structure. Please refer to Figure 8 , Figure 8 It is a schematic diagram of the Uformer network structure provided by the embodiments of the present invention. As shown in the figure, the Uformer network structure in this embodiment includes: an input mapping module, a plurality of encoder modules, a plurality of decoder modules, and an output mapping module.

[0062] Among them, the input mapping module is used to perform primary feature extraction on the input multi-frame images to obtain a primary feature map. In this embodiment, a 3×3 convolutional layer with LeakyReLU is used to extract primary features from the input multi-frame images to obtain a primary feature map.

[0063] Further, the encoder module includes cascaded LeWin Transformer blocks and a downsampling layer. The encoder module is used to perform feature extraction and fusion on the primary feature map to obtain a fused feature map; the decoder module includes cascaded upsampling layers and LeWin Transformer blocks. The decoder module is used to perform learning and restoration on the fused feature map to obtain a one-dimensional feature map. In this embodiment, the upsampling layer and the downsampling layer serve as feature extraction modules. The LeWin Transformer block in the decoder module learns and restores the image according to the obtained features. The features obtained by the LeWin Transformer block in the decoder module are the upsampled features and the corresponding features obtained from the LeWin Transformer block in the encoder through skip connections.

[0064] Further, the output mapping module is used to perform feature mapping on the one-dimensional feature map to obtain a restored image. In this embodiment, the output mapping module reshapes the input one-dimensional feature map into a two-dimensional feature map and applies a 3×3 convolutional layer to obtain the restored image.

[0065] The Uformer network structure of the deep learning model constructed in this embodiment has a basic structure of U-Net and combines the transformer. The convolutional blocks in U-Net are modified into transformer blocks. On the one hand, it can utilize the advantages of the skip connection structure to fuse multi-scale information, and on the other hand, it can utilize the attention mechanism in the transformer to capture global dependencies, so as to obtain better results.

[0066] Further, the LeWin Transformer block in this embodiment is a locally enhanced window transformer block, as Figure 9 shown Figure 9 FIG. is a schematic structural diagram of a LeWin Transformer block provided by an embodiment of the present invention. The LeWin Transformer block includes a window-based multi-head self-attention module (W-MSA) and a locally enhanced feed-forward network (LeFF). Among them, the window-based multi-head self-attention module uses the self-attention mechanism to capture global dependencies, and the locally enhanced feed-forward network uses depth convolution to capture local information. As shown in the figure, the LeWin Transformer block in this embodiment also includes a normalization layer and an addition layer, and their functions are to normalize and add the extracted features respectively.

[0067] In this embodiment, the LeWin Transformer block reduces the computational cost by using the self-attention mechanism of non-overlapping windows on the feature map. Moreover, the W-MSA module does not use global attention like ordinary transformers, but uses it locally in a non-overlapping manner, which can not only greatly reduce the computational amount, but also obtain the neighborhood information of degraded pixels, which is crucial for the image restoration task; the LeFF module uses depth convolution to capture local information and can better utilize context information.

[0068] Step 2b: Generate a training data set.

[0069] In this embodiment, the training data set is generated by a simulation experiment in a dynamic matrix transformation manner.

[0070] (1) Simulation experiment conditions:

[0071] The hardware platform for the simulation experiment is as follows: the processor is an Intel Core i7-9700K with a frequency of 3.6 Hz, and the graphics card is an NVIDIA TITAN Xp.

[0072] The software platform for the simulation experiment is as follows: Windows 10 operating system and Python 3.7.

[0073] The dataset used in the simulation is NWPU-RESISC45, which is a Remote Sensing Image Scene Classification (RESISC) dataset created by Northwestern Polytechnical University (NWPU). It includes 31,500 images in 45 scene classes such as airplanes, airports, baseball fields, basketball courts, beaches, bridges, churches, business districts, deserts, and forests. Each class has 700 images, and the image size is 256*256. In this embodiment, only 500 images in 10 classes are used in the experiment.

[0074] (2) Simulation content

[0075] Simulation experiments were conducted on three systems, including the optical synthetic aperture dynamic variable array system proposed in this embodiment and two sets of comparative experiments (a traditional Y-shaped system with a fill factor of 0.11 and a six-aperture annular system with a fill factor of 0.2). Gaussian noise with a mean of 0 and a variance of 0.0001 was added to all three systems.

[0076] Step 2c: Train the network;

[0077] In this embodiment, the number of encoders and decoders in the network is set to 4. The network optimizer uses AdamW, the momentum terms are (0.9, 0.999), the weight decay is 0.02, the initial learning rate is 2e-4, and the cosine decay strategy is used to reduce it to 1e-6. The window size of the LeWin Transformer block is set to 8×8, and training is carried out for 250 epochs with a batch size of 1. The loss function is as follows, and the evaluation metrics use image quality evaluation metrics: PSNR and SSIM:

[0078]

[0079] In the formula, is the ground-truth image, and R is the restored image.

[0080] Step 2d: Test and verify the trained deep learning model.

[0081] Specifically, the trained deep learning model is used to restore the degraded image. In this embodiment, to compare and verify the restoration ability of the deep learning model (Uformer) of this embodiment, the traditional algorithm Wiener filtering and the basic network U-Net are used as reference groups respectively.

[0082] Please refer to Figure 10 , Figure 10 which is a comparison chart of the different network restoration effects of the Y-type and ring-type array systems provided by the embodiments of the present invention. Among them, figure (a) is the original input image, figure (b) is the observed image obtained after passing through the optical synthetic aperture system, and it can be clearly seen that the clarity of the image decreases and details are lost; figure (c) is the result obtained by using the multi-frame Wiener filtering algorithm. Although the blurring phenomenon is effectively suppressed, there will be varying degrees of ringing effects at the object edges and image edges, which is related to the difficulty of estimating the signal and noise power spectra; figure (d) is the restored image obtained by using the U-Net network. After network restoration, the image clarity is improved and the ringing effect is alleviated. It can be seen that network restoration can eliminate the influence of noise and the degradation process; figure (e) is the restored result obtained by using the deep learning model (Uformer) of this embodiment. Compared with the restoration of U-Net, the details are more obvious.

[0083] The evaluation indicators are used to evaluate the restored results obtained by the two systems through three restoration methods respectively, and the results are shown in Table 1.

[0084] Table 1 Comparison of indicators of the test set under different systems

[0085]

[0086] It can be seen from Table 1 that the restoration result of the U-Net network has been greatly improved compared with Wiener filtering. The PSNR and SSIM have increased by at least 3dB and 0.1 respectively. Comparing the restoration results of the U-Net network and Uformer, in the Y-type system, the PSNR of the restored image of Uformer has increased by 3.41dB compared with the U-Net network, and the SSIM has increased by 0.049. In the ring-type system, the PSNR of the restored image of Uformer has increased by 2.1dB compared with the U-Net network, and the SSIM has increased by 0.028. The restoration indicators have been significantly improved.

[0087] Furthermore, the traditional Wiener filtering algorithm and the deep neural network are respectively used to conduct experimental verification on the optical synthetic aperture dynamic variable array of this embodiment.

[0088] Please refer to Figure 11 , Figure 11It is a comparison diagram of the visual effects of restored images of different imaging methods provided by the embodiments of the present invention. As shown in the figure, the figure shows the enlarged partial details of the images restored by two different algorithms for two systems. It can be seen that the image restored by Wiener filtering has a lot of noise. This is because for the variable array system, the multi-frame Wiener filtering restoration algorithm restores the image to be restored by stacking it up. At this time, the noise will also be stacked, and the influence of the noise will be amplified. Therefore, for the restoration of multi-frame images, Wiener filtering is not the best solution, which is also the limitation of traditional restoration algorithms. On the contrary, the results of network restoration have been greatly improved, and the vision is clearer. This is because in the process of feature extraction and restoration, the noise is eliminated to a certain extent, and the rich information of multi-frame images can also be fully utilized. The evaluation indicators are used to evaluate the restoration results obtained by the two systems through the two restoration methods respectively, and the results are shown in Table 2.

[0089] Table 2 Comparison of indicators of the test set under different variable array systems

[0090]

[0091] The optical synthetic aperture dynamic variable array imaging system of this embodiment is provided with an optical synthetic aperture dynamic variable array, which realizes the imaging effect of an equivalent large aperture with a small aperture sparse system through a variable array method of rotation and expansion, and restores the multi-frame observation image group generated by the system through a deep learning algorithm, so as to achieve high-resolution imaging. In addition, the optical synthetic aperture dynamic variable array with an asymmetric structure proposed in this embodiment breaks through the traditional axisymmetric or centrosymmetric method, and broadens the research direction.

[0092] Embodiment 2

[0093] This embodiment provides an optical synthetic aperture dynamic variable array imaging method, which is applicable to the optical synthetic aperture dynamic variable array imaging system described in Embodiment 1. Please refer to Figure 12 , Figure 12 It is a flowchart of an optical synthetic aperture dynamic variable array imaging method provided by the embodiments of the present invention. As shown in the figure, the optical synthetic aperture dynamic variable array imaging method of this embodiment includes:

[0094] Step 1: Use a four-aperture array with an X-shaped baseline as the initial array, where the angle of the X-shaped baseline is π / 8, the X-shaped baseline formed by the four-aperture array is asymmetric, and the difference between the longer baseline and the shorter baseline is equal to the diameter of the sub-aperture;

[0095] Step 2: Change the baseline length with a telescopic step of 50 mm, and at the same time use a detector to collect the images of the object to be measured corresponding to different baseline lengths, where the diameter of the circumscribed circle of the four-aperture array does not exceed 1 m;

[0096] Step 3: Rotate the four-aperture array one full circle with a rotation angle of π / 8, and repeat Step 2 after each rotation to obtain multiple frames of images of the object to be measured;

[0097] Step 4: Input the multiple frames of images into the trained deep learning model to obtain the restored image.

[0098] In this embodiment, the specific construction and training process of the deep learning model is similar to that in Embodiment 1, and will not be elaborated here.

[0099] The optical synthetic aperture dynamic variable array imaging method of this embodiment restores the optical synthetic aperture image through deep learning, combines deep learning and the variable array system, and combines the multi-channel characteristics of deep learning with the multiple frames of images of the variable array system, breaking through the problem of difficult processing of multiple frames of images by traditional algorithms.

[0100] It should be noted that in this article, the terms "including", "comprising" or any other variant are intended to cover non-exclusive inclusion, so that an article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed. Without more limitations, an element defined by the statement "including one..." does not exclude the existence of another identical element in the article or device including the said element. "Connection" or "connected" and other similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect.

[0101] The above content is a further detailed description of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is only limited to these descriptions. For those of ordinary skill in the technical field to which the present invention belongs, without departing from the concept of the present invention, several simple deductions or substitutions can be made, and all should be regarded as belonging to the protection scope of the present invention.

Claims

1. An optical synthetic aperture dynamic variable array imaging system, characterized in that, it includes: a multi-frame image acquisition module and an image restoration module, wherein, the multi-frame image acquisition module includes an optical synthetic aperture dynamic variable array and a detector connected in sequence, wherein the optical synthetic aperture dynamic variable array includes a plurality of sub-apertures, and the plurality of sub-apertures form an aperture array with an X-shaped baseline, and the dynamic variable array method of rotating the aperture array and stretching and contracting the baseline is used to realize the multi-frame image acquisition of the object to be measured by the detector; the baseline is the connection line between the center of the circumcircle of the aperture array and the center of the sub-aperture; the aperture array is a four-aperture array with an X-shaped baseline, and during the dynamic variable array process, the diameter of the circumcircle of the four-aperture array does not exceed 1 m; during the dynamic variable array process, the aperture array rotates within an angle range of 0 to 2π, and the rotation angle is π / 8 - π / 2; during the dynamic variable array process, the stretching and contracting step length of the baseline is 40 mm - 70 mm; the X-shaped baseline formed by the aperture array is asymmetric, wherein the difference between the longer baseline and the shorter baseline is equal to the diameter of the sub-aperture; the image restoration module uses the trained deep learning model to restore the multi-frame images to obtain restored images, wherein the deep learning model adopts the Uformer network structure, and the Uformer network structure includes: an input mapping module, a plurality of encoder modules, a plurality of decoder modules, and an output mapping module, wherein, the input mapping module is used for primary feature extraction of the input multi-frame images to obtain a primary feature map; the encoder module includes a cascaded LeWin Transformer block and a downsampling layer, and the encoder module is used for feature extraction and fusion of the primary feature map to obtain a fused feature map; the decoder module includes a cascaded upsampling layer and a LeWin Transformer block, and the decoder module is used for learning and restoration of the fused feature map to obtain a one-dimensional feature map; the output mapping module is used for feature mapping of the one-dimensional feature map to obtain the restored image.

2. The optical synthetic aperture dynamic variable array imaging system according to claim 1, characterized in that, the diameter of the sub-aperture is 50 - 100 mm; the angle range of the X-shaped baseline formed by the aperture array is π / 8 - π / 2.

3. The optical synthetic aperture dynamic variable array imaging system according to claim 1, characterized in that, the LeWin Transformer block is a local enhanced window transformer block, including a window-based multi-head self-attention module and a local enhanced forward network, wherein the window-based multi-head self-attention module uses the self-attention mechanism to capture global dependencies, and the local enhanced forward network uses depth convolution to capture local information.

4. An optical synthetic aperture dynamic variable array imaging method, characterized in that, it includes: Step 1: Use a four-aperture array with an X-shaped baseline as the initial array. Here, the angle of the X-shaped baseline is π / 8, and the X-shaped baseline formed by the four-aperture array is asymmetric. The difference between the longer baseline and the shorter baseline is equal to the diameter of the sub-aperture. The baseline is the line connecting the center of the circumcircle of the aperture array and the center of the sub-aperture. Step 2: Change the baseline length with a telescopic step of 50 mm. At the same time, use a detector to collect images of the object to be measured corresponding to different baseline lengths. Here, the diameter of the circumcircle of the four-aperture array does not exceed 1 m. Step 3: Rotate the four-aperture array one week with an angle of π / 8. After each rotation, repeat Step 2 to obtain multiple frames of images of the object to be measured. Step 4: Input the multiple frames of images into the trained deep learning model to obtain a restored image. Among them, the deep learning model adopts the Uformer network structure. The Uformer network structure includes: an input mapping module, several encoder modules, several decoder modules, and an output mapping module. Among them, The input mapping module is used to perform primary feature extraction on the input multiple frames of images to obtain a primary feature map. The encoder module includes a cascaded LeWin Transformer block and a downsampling layer. The encoder module is used to perform feature extraction and fusion on the primary feature map to obtain a fused feature map. The decoder module includes a cascaded upsampling layer and a LeWin Transformer block. The decoder module is used to perform learning and restoration on the fused feature map to obtain a one-dimensional feature map. The output mapping module is used to perform feature mapping on the one-dimensional feature map to obtain a restored image.

Images