Method for inverting misadjustment of optical system by point spread function pattern

Abstract

This invention relates to the field of on-orbit telescope setup and adjustment technology, specifically providing a method for inverting the offset of an optical system using point spread function patterns. The method includes the following steps: constructing an optical model of the optical system to be set up and adjusted; applying a preset offset; acquiring corresponding star patterns in multiple preset fields of view; labeling the corresponding offset and field of view on each star pattern; and constructing a training dataset. A setup and adjustment prediction network based on a generative adversarial network for auxiliary classification is established, and the network is trained using the training dataset. The trained network is then used to estimate the offset and perform system correction on the optical system to be set up and adjusted. This invention eliminates the need for additional wavefront sensors, directly inverting active correction values ​​using star patterns acquired by an on-orbit camera.

Description

Technical Field

[0001] This invention belongs to the field of on-orbit telescope assembly and adjustment technology, and particularly relates to a method for retrieving optical system misalignment from point spread function patterns. Background Technology

[0002] During on-orbit operation, optical remote sensing systems are susceptible to spatial pose shifts in their optical components due to factors such as system deployment, vibration and shock, stress release from their own structures, and changes in the gravitational environment. This can lead to system misalignment and consequently, a decline in image quality. Therefore, real-time monitoring and precise correction of the system's on-orbit status are necessary.

[0003] Existing on-orbit assembly and adjustment methods for space telescopes mainly fall into three categories: wavefront-based, image evaluation function-based, and deep learning-based. Among them, wavefront-based methods struggle to acquire aberration information under on-orbit conditions; evaluation function-based methods suffer from numerous iterations, slow convergence, and high mechanical losses; while existing deep learning methods are mostly designed for coaxial or finite-degree-of-freedom systems, making it difficult to adapt to the on-orbit assembly and adjustment requirements of complex optical systems with coupled surface shape and pose errors and asymmetric aberrations across the entire field of view.

[0004] Therefore, it is necessary to propose an intelligent on-orbit solution method that does not require additional wavefront sensors, directly inverts active correction quantities using star patterns acquired by on-orbit cameras, and is applicable to camera pose coupling errors. Summary of the Invention

[0005] In view of this, the present invention aims to provide a method for inverting the offset of an optical system using a point spread function pattern. By introducing the offset as a conditional constraint, a nonlinear mapping relationship is established between the point spread pattern and the system offset parameters, thereby achieving high-precision calculation under the condition of pose error coupling in the optical system. This solves the drawbacks of traditional schemes that rely on wavefront detection or multiple iterations, and enables rapid on-orbit assembly and adjustment of the camera system.

[0006] To achieve the above objectives, the technical solution created by this invention is implemented as follows: This invention provides a method for retrieving the misalignment of an optical system from a point spread function pattern, comprising: S1: Construct the optical model of the optical system to be installed and adjusted; S2: Apply a preset offset amount to the optical model of the optical system to be installed and adjust, and obtain the corresponding star point patterns in multiple preset fields of view. Mark the corresponding offset amount and field of view on each star point pattern, and process the multi-field star point patterns into point spread function patterns to form a training dataset. S3: Establish an assembly and adjustment prediction network based on an auxiliary classification generative adversarial network. The assembly and adjustment prediction network includes a generator and a discriminator. The generator is configured to output the corresponding point spread function prediction pattern based on the input offset, field of view and random noise. The discriminator is configured to: take a point spread function prediction pattern or a point spread function pattern as input, identify and output the category of the input pattern and its corresponding misalignment estimate; S4: Use the training dataset to perform adversarial training on the assembly prediction network to learn the mapping relationship between the multi-viewpoint spread function pattern and the misalignment. S5: Input the measured spread function pattern of the optical system to be assembled and adjusted in actual work into the trained assembly and adjustment prediction network. The assembly and adjustment prediction network outputs the misalignment corresponding to the measured spread function pattern, and the optical system to be assembled and adjusted is corrected based on the misalignment.

[0007] Preferably, the optical system to be installed and adjusted is an off-axis three-mirror telescope system, which includes a primary mirror, a secondary mirror, a third mirror, and a folding mirror.

[0008] Preferably, in S1, an optical model of the optical system to be assembled and adjusted is constructed using optical design software, and the structural parameters and positions of each component in the optical system to be assembled and adjusted, as well as the spatial pose relationship between the components, are set.

[0009] Preferably, the preset field of view includes: a central field of view and at least one off-axis edge field of view.

[0010] Preferably, processing the multi-field star pattern into a point spread function pattern includes: The star pattern is subjected to noise reduction, background removal, and grayscale normalization. Threshold segmentation or extreme value detection methods are used to locate the stars in the star pattern and extract local sub-images centered on the stars; Subpixel registration and center alignment are performed on local sub-images to eliminate positional deviations; The point spread function pattern of the corresponding field of view is obtained by reconstructing the energy distribution of star points through Gaussian fitting, interpolation, or neural network methods.

[0011] Preferably, in S4, the generator is used to train and learn the positive mapping relationship between the offset, the field of view, and random noise to generate the point spread function prediction pattern; the discriminator is used to learn to distinguish whether the input pattern is a point spread function prediction pattern or a point spread function pattern, and generate the corresponding offset estimate.

[0012] Compared with the prior art, the present invention can achieve the following beneficial effects: This invention uses Auxiliary Classification Generative Adversarial Network (ACGAN) to directly invert optical system misalignment from star patterns acquired by on-orbit cameras, without the need for additional wavefront sensors. This reduces the hardware complexity of space optical systems and overcomes the design bottleneck of difficulty in obtaining aberration information under on-orbit conditions.

[0013] This invention learns the positive mapping relationship between offset and star pattern through a generator, generating a star pattern that approximates the true distribution. A discriminator identifies the authenticity of the input pattern and determines its corresponding offset. An end-to-end neural network from PSF pattern to offset is constructed. Compared with existing evaluation functions, it avoids problems such as convergence difficulty and large mechanical loss. Compared with existing deep learning methods, this invention can handle field-dependent asymmetric aberrations and inter-element error coupling in off-axis systems through multi-field coverage and adversarial training, and can be applied to complex optical systems such as off-axis systems.

[0014] During ground assembly and adjustment, this invention utilizes a collimator to simulate and collect measured star point samples from multiple fields of view. This data, combined with simulation data, forms a training set to complete the ground training of the ACGAN model. During on-orbit operation, the trained discriminator is deployed in the optical system. Inputting actual captured star point images outputs the offset, driving the actuator to complete rapid assembly and image quality restoration. Compared to existing evaluation function methods, the ACGAN model does not require multiple iterations; a single inference is sufficient to output the offset, meeting on-orbit adjustment requirements. Attached Figure Description

[0015] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments and descriptions of the invention are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings: Figure 1 This is a flowchart of a method for retrieving optical system misalignment from a point spread function pattern according to an embodiment of the present invention; Figure 2 This is a schematic diagram of an off-axis three-mirror optical system provided according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of an ACGAN network provided according to an embodiment of the present invention; Figure 4 This is a schematic diagram of the field of view distribution for star point pattern acquisition according to an embodiment of the present invention. Detailed Implementation

[0016] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only for explaining the invention and do not constitute a limitation thereof. Similar elements in different embodiments are referred to by associated similar element reference numerals. In the following embodiments, many details are described to facilitate a better understanding of the invention. However, those skilled in the art will readily recognize that some features may be omitted in different situations, or may be replaced by other elements, materials, or methods. In some cases, some operations related to the invention are not shown or described in the specification. This is to avoid obscuring the core parts of the invention with excessive description. For those skilled in the art, detailed description of these related operations is not necessary; the relevant operations can be fully understood based on the description in the specification and general technical knowledge in the art.

[0017] It should be noted that, unless otherwise specified, the embodiments and features described in this invention can be combined to form various implementations. Furthermore, the order of the steps or actions in the method description can be changed or adjusted in a manner readily apparent to those skilled in the art. Therefore, the various orders in the specification and drawings are merely for the clear description of a particular embodiment and do not imply a mandatory order, unless otherwise stated that a particular order must be followed.

[0018] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," and "counterclockwise," etc., indicating orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation on this invention. Furthermore, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, features defined with "first," "second," etc., may explicitly or implicitly include one or more of that feature. In the description of this invention, unless otherwise stated, "a plurality of" means two or more.

[0019] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art will understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0020] The invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0021] Please see Figure 1 In one embodiment of the present invention, a method for retrieving the misalignment of an optical system from a point spread function pattern is provided. This method eliminates the need for an additional wavefront sensor. During the ground-based assembly and adjustment experiment, an assembly and adjustment prediction network based on an auxiliary classification generative adversarial network is constructed and trained. Misalignment is introduced to generate a large number of training samples, which are then used to train the network. The trained network is then deployed on an on-orbit camera. During the camera's on-orbit operation, the multi-field star pattern under the condition to be assembled and adjusted is acquired using the camera system's built-in focal plane detector. This pattern is then input into the trained network, directly outputting the adjustment amount of the component to be corrected, thereby achieving rapid on-orbit assembly and adjustment of the camera system. The method of the present invention specifically includes: S1: Construct the optical model of the optical system to be installed and adjusted; S2: Apply a preset offset amount to the optical model of the optical system to be installed and adjust, and obtain the corresponding star point patterns in multiple preset fields of view. Mark the corresponding offset amount and field of view on each star point pattern, and process the multi-field star point patterns into point spread function patterns to form a training dataset. S3: Establish an assembly and adjustment prediction network based on an auxiliary classification generative adversarial network. The assembly and adjustment prediction network includes a generator and a discriminator. The generator is configured to output the corresponding point spread function prediction pattern based on the input offset, field of view and random noise. The discriminator is configured to: take a point spread function prediction pattern or a point spread function pattern as input, identify and output the category of the input pattern and its corresponding misalignment estimate; S4: Use the training dataset to perform adversarial training on the assembly prediction network to learn the mapping relationship between the multi-viewpoint spread function pattern and the misalignment. S5: Input the measured spread function pattern of the optical system to be assembled and adjusted in actual work into the trained assembly and adjustment prediction network. The assembly and adjustment prediction network outputs the misalignment corresponding to the measured spread function pattern, and the optical system to be assembled and adjusted is corrected based on the misalignment.

[0022] In step S1, an optical model of the optical system to be installed and adjusted is first constructed. This optical system is an off-axis three-mirror optical system for on-orbit application, such as... Figure 2 As shown, the optical components of the off-axis three-mirror optical system include a primary mirror, a secondary mirror, a tertiary mirror, and a folding mirror. A folding mirror is introduced into the system's optical path to change the system's light output direction, forming a representative asymmetric spatial optical path structure. The main function of the folding mirror is to change the system's light output direction to meet the overall layout and engineering application requirements. This system has a complex structure and strong coupling relationships between components. A schematic diagram of the system's optical layout is shown in the figure. The primary mirror and the tertiary mirror are both partial sections of even-order aspherical surfaces. Their optical surface shapes exhibit significant truncation characteristics compared to a complete symmetrical aspherical surface, causing the effective optical axis of the mirror surface to not coincide with the mechanical symmetry axis. During assembly and adjustment, the spatial pose (including translation and tilt) of these two mirrors is highly sensitive to the system's image quality; even small assembly and adjustment errors can cause significant aberration changes, placing stringent requirements on the accuracy and stability of the assembly and adjustment algorithm. The secondary mirror also adopts an even-order aspherical form, and its optical axis direction is defined as the system's global optical axis direction, playing a crucial optical axis reference role in the system. The accuracy of the secondary mirror's pose directly affects the stability of the system's optical axis and the relative alignment between the various mirrors, making it one of the core control objects during the assembly and adjustment process.

[0023] During the ground commissioning phase, a ground-based experimental platform for the off-axis three-mirror optical system needs to be built. Using optical simulation software, an optical model of the off-axis three-mirror optical system is constructed based on this platform. The structural parameters of each optical element and the spatial pose relationships between them are set, serving as the foundation for subsequent misalignment modeling and imaging simulation. The secondary mirror is used as a reference reference during the determination of spatial pose relationships. Furthermore, the positional parameters of each optical element need to be calibrated to provide a unified reference for introducing misalignment and multi-field imaging simulation. Further, based on the optical model of the optical system to be assembled and adjusted, multi-field imaging performance analysis is performed to obtain the PSF pattern distribution characteristics of the system under ideal assembly and adjustment conditions, providing a foundation for subsequent misalignment modeling and dataset construction.

[0024] In step S2, by introducing offset as a constraint, a training dataset containing star patterns from multiple fields of view and corresponding system offsets is constructed. A nonlinear mapping relationship between the star patterns and system offset parameters is established, enabling high-precision calculation under pose error coupling conditions in the optical system. Specifically, a preset offset is applied to the optical model of the optical system to be fitted, placing the optical system in a preset error state. The target of applying the preset offset is the optical elements in the optical system, including adjusting the eccentricity of one or more optical elements and pose disturbances such as tilting around orthogonal coordinate axes. After introducing element offsets, imaging simulations or acquisitions are performed in multiple preset fields of view to obtain star patterns from different fields of view. The preset fields of view include a central field of view and at least one off-axis edge field of view to reflect the asymmetric aberration distribution characteristics of the off-axis three-mirror optical system under different fields of view. Figure 4 As shown, in this embodiment of the invention, the preset field of view includes a central field of view 1, an edge field of view 2, and an edge field of view 3. For each obtained star pattern, it is necessary to label its corresponding optical element misalignment and field of view, and establish a mapping relationship between the star pattern and the system misalignment.

[0025] After acquiring multi-field star point images, in order to accurately invert the misalignment of the optical system, the original, noisy, misaligned, and discretely sampled star point images need to be transformed into clean, aligned, continuous point spread function (PSF) images that only reflect the intrinsic aberrations of the optical system. This requires preprocessing the original star point images and inverting the PSF images for the corresponding fields of view. Specifically: First, the original star images are denoised. Since star images acquired in orbit or on the ground generally contain noise, filtering algorithms can be used to suppress the noise while preserving the peak shape of the star energy distribution and removing the background of the stars. Finally, the image is normalized to eliminate the differences in absolute brightness caused by different exposure times or magnitudes.

[0026] Secondly, methods such as threshold segmentation or extreme value detection are used to locate the star points in the star point image, and local sub-images centered on the star points are extracted. Sub-pixel registration and center alignment processing are performed on the local sub-images. In the processed local sub-images, the star point peaks are strictly located at the center pixel position of the sub-image, thereby eliminating the influence of positional deviation.

[0027] Finally, the energy distribution of star points is reconstructed by Gaussian fitting, interpolation, or neural network methods to obtain the predicted point spread function pattern under the corresponding field of view.

[0028] Based on the above data organization method, a standardized training dataset covering multiple fields of view and multiple error combinations can be constructed to characterize the imaging response characteristics of the optical system under different misalignment states, providing a data foundation for subsequent network training.

[0029] In step S3, establish as follows Figure 3 The diagram illustrates an assembly prediction network based on an Auxiliary Classification Generative Adversarial Network (ACGAN). This network comprises a generator and a discriminator. The generator takes random noise and conditional label information as input. The random noise is used to introduce diversity into the generated samples, and the conditional labels are the offset and field of view of the optical system. The generator outputs a predicted point spread function (PSF) pattern for the corresponding label state by performing a forward mapping from the offset, field of view, and random noise to the point spread pattern. This predicted pattern is a simulated pseudo-PSF pattern. The generator can simulate the imaging distribution under different offset states and learn the forward mapping relationship. During training, the generator attempts to generate images that are consistent with the real PSF pattern distribution and meet the preset offset to deceive the discriminator.

[0030] The discriminator takes either a true PSF pattern (i.e., a point spread function pattern) or a pseudo PSF pattern (i.e., a predicted point spread function pattern) output by the generator as input. The discriminator determines the category of the input pattern, specifically whether it is a predicted point spread function pattern or a true point spread function pattern. One output of the discriminator is a binary probability of the input pattern being "true" or "false," used for adversarial training. The other output is an estimate of the optical system offset corresponding to the input pattern, enabling a direct regression from the PSF pattern to the physical offset. The discriminator's dual outputs provide a joint constraint on the sample category (true / false) and the offset, forcing the generator to adhere to a given offset condition while generating realistic patterns.

[0031] In step S4, the training dataset constructed in step S2 is divided into a training set, a validation set, and a test set. The assembly prediction network constructed in step S3 is trained using the training set. Training samples from the training dataset are input into the assembly prediction network. The generator learns the positive mapping relationship between the offset, field of view, and random noise to the point spread function (PSF) pattern, generating a PSF prediction pattern that approximates the true distribution. The PSF prediction pattern generated by the generator and the PSF pattern from the training dataset are merged into a single dataset and input into a discriminator. The discriminator performs a true / false judgment based on the input PSF prediction pattern and the PSF pattern, regresses the corresponding offset, and outputs the corresponding offset estimate.

[0032] This invention, through the introduction of adversarial loss, offset regression loss, and reference imaging-based constraint terms during training, jointly optimizes the generator and discriminator, enabling the model to accurately represent the complex nonlinear mapping relationship between the multi-viewpoint spread function pattern and the system offset. During training, the generator and discriminator continuously adjust their parameters to minimize the adversarial loss function. The generator's loss function comprises both adversarial and class loss, while the discriminator's loss function comprises both adversarial and classification loss. By optimizing the adversarial loss, the generator and discriminator can continuously improve their performance, thereby generating more realistic images that conform to the class labels.

[0033] In step S6, following the training process in step S5, the assembly and adjustment prediction network obtained in step S5 is validated and tested using samples from the validation and test sets. Specifically, point spread function patterns not involved in network training are input into the assembly and adjustment prediction network trained in step S5 to obtain estimated values ​​of the pose misalignment of the optical element to be adjusted. The estimated misalignment values ​​are compared with the corresponding actual misalignment values, and statistical evaluation is performed to characterize the model's solution accuracy and stability. After validation and testing, the trained assembly and adjustment prediction network is deployed on the optical system. Once the optical system is in orbit, the star patterns acquired in real-time by the focal plane detector of the optical system are preprocessed into point spread function patterns and input into the assembly and adjustment prediction network deployed on the optical system. The assembly and adjustment prediction network outputs real-time misalignment estimates and generates corresponding optical system adjustment data based on these estimates. This optical system adjustment data is then input into the drive actuator to correct the optical system to be assembled, completing rapid assembly and image quality restoration.

[0034] In summary, the above description is merely a preferred embodiment of this specification and is not intended to limit the scope of protection of this specification. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of protection of this specification.

[0035] The systems, apparatuses, modules, or units described in one or more of the above embodiments may be implemented by a computer chip or entity, or by a product having a certain function. A typical implementation device is a computer. Specifically, a computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or any combination of these devices.

[0036] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0037] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

Claims

1. A method for retrieving misalignment of an optical system from a point spread function plot, characterized in that, include: S1: Construct the optical model of the optical system to be installed and adjusted; S2: Apply a preset offset amount to the optical model of the optical system to be installed and adjust, and obtain corresponding star patterns in multiple preset fields of view. Mark the corresponding offset amount and field of view on each star pattern, and process the multi-field star pattern into a point spread function pattern to form a training dataset. S3: Establish an assembly and adjustment prediction network based on an auxiliary classification generative adversarial network, wherein the assembly and adjustment prediction network includes a generator and a discriminator; wherein the generator is configured to output a corresponding point spread function prediction pattern based on the input offset, field of view and random noise; The discriminator is configured to: take a point spread function prediction pattern or a point spread function pattern as input, and determine and output the category of the input pattern and its corresponding misalignment estimate; S4: Use the training dataset to perform adversarial training on the assembly prediction network to learn the mapping relationship between the multi-viewpoint spread function pattern and the misalignment. S5: Input the measured point spread function pattern obtained from the actual operation of the optical system to be assembled and adjusted into the assembled and adjusted prediction network after training. The assembled and adjusted prediction network outputs the misalignment corresponding to the measured point spread function pattern, and corrects the optical system to be assembled and adjusted based on the misalignment.

2. The method for retrieving optical system misalignment from point spread function patterns according to claim 1, characterized in that, The optical system to be installed and adjusted is an off-axis three-mirror telescope system, which includes a primary mirror, a secondary mirror, a third mirror, and a folding mirror.

3. The method for retrieving optical system misalignment from point spread function patterns according to claim 1, characterized in that, In step S1, an optical model of the optical system to be assembled and adjusted is constructed using optical design software, and the structural parameters and positions of each component in the optical system to be assembled and adjusted, as well as the spatial pose relationship between the components, are set.

4. The method for retrieving optical system misalignment from point spread function patterns according to claim 1, characterized in that, The preset field of view includes: a central field of view and at least one off-axis edge field of view.

5. The method for retrieving optical system misalignment from point spread function patterns according to claim 1, characterized in that, The process of converting a multi-field star point pattern into a point spread function pattern includes: The star pattern is subjected to noise reduction, background removal, and grayscale normalization. Threshold segmentation or extreme value detection methods are used to locate the stars in the star pattern and extract local sub-images centered on the stars; Subpixel registration and center alignment are performed on the local sub-image to eliminate positional deviations; The point spread function pattern of the corresponding field of view is obtained by reconstructing the energy distribution of star points through Gaussian fitting, interpolation, or neural network methods.

6. The method for retrieving optical system misalignment from point spread function patterns according to claim 1, characterized in that, In step S4, the generator is used to train and learn a positive mapping relationship between the offset, the field of view, and random noise to generate a point spread function prediction pattern; the discriminator is used to learn to distinguish whether the input pattern is a point spread function prediction pattern or a point spread function pattern, and generate the corresponding offset estimate.

Images