Calculation processing platform giving consideration to real-time performance and flexibility of AO system
By combining the processing architecture of FPGA and multi-core CPU in an adaptive optical system, the slope extraction and voltage recovery tasks are completed, and the system delay and flexibility requirements are solved, efficient real-time processing and flexible control are achieved, and the imaging quality and closed-loop performance of the AO system are improved.
Patent Information
- Application Number
- CN202510011033.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-03
- Publication Date
- 2025-05-06
AI Technical Summary
The existing real-time processors of adaptive optical systems handle high bandwidth and dynamic changes in atmospheric conditions, excessive delays lead to reduced closed-loop bandwidth in the system, affecting imaging quality, and traditional FPGA and DSP architectures are difficult to meet flexibility requirements.
FPGA is used to complete the slope extraction task with large calculation volume and fixed algorithms. The multi-core CPU completes voltage restoration and control operations with small calculation volume and frequent algorithm changes. The Hartman image sub-aperture row transfer reduces the transmission delay of the entire frame of image data, and combines the encoding flexibility of the multi-core CPU to achieve a balance of real-time and flexibility.
It effectively reduces data transmission delay, ensures the real-time nature of the system, and achieves the best correction effect through flexible control algorithm adjustment, improving the closed-loop dynamic performance of the solar AO system.
Smart Images
Figure CN119938586A_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the field of adaptive optics, and in particular relates to a computing and processing platform that takes into account both the real-time performance and flexibility of an AO system. Background Art
[0002] Adaptive optics (AO) is an indispensable technology for large-aperture ground-based telescopes to achieve high-resolution imaging. The real-time processor is the computing core of the entire AO system. It is located after the wavefront sensor and before the wavefront controller. It converts the specific distortion information detected by the sensor into a voltage that can directly act on the corrector, playing the role of transmission conversion. Its conversion process involves large-scale data calculation. The main function of the AO system is to correct the wavefront distortion caused by factors such as atmospheric turbulence in real time. If the delay is too high, the corrector cannot respond to the control voltage calculated by the processor before the next frame of the image is exposed. At this time, the image collected by the Hartmann camera is repeated with the previous frame, which reduces the closed-loop bandwidth of the system and affects the imaging quality. On the one hand, the solar AO system is limited by the worse atmospheric seeing conditions during the day, requiring a higher system effective bandwidth, which requires the real-time processor to have stronger computing power and shorter processing delay; on the other hand, the solar AO system needs to use low-contrast extended targets (sunspots, rice grains, etc.) on the surface of the sun as beacons for wavefront detection. The extended target wavefront detection based on the relevant algorithm also further increases the computing burden of the real-time processor. Therefore, the processing performance and computational delay of the real-time processor directly determine the closed-loop dynamic performance of the solar AO system.
[0003] In order to reduce latency, AO systems mostly use various high-speed processing units and their combinations as real-time processor architectures. Traditional AO real-time processor architectures usually use dedicated devices such as FPGA and DSP to meet the real-time requirements of the system. With the gradual enhancement of the parallel computing capabilities of commercial graphics processing units (GPUs) and multi-core central processing units (CPUs), GPUs and multi-core CPUs have gradually become research hotspots for the real-time processor architecture of AO systems. With the development of adaptive optics technology, the expansion of scale and the enrichment of application fields, for solar AO systems, not only is the processor latency required to be low, but the processor control algorithm also needs to change accordingly with the changes in atmospheric conditions to achieve the best correction effect. Common control algorithms include: proportional integral (PI) control, predictive control, optimal control, etc. The PI control algorithm is relatively mature in industrial applications and has a simple algorithm. However, for an atmospheric environment with drastic dynamic changes, the control algorithm cannot predict the turbulence changes after time delay, resulting in the inability to achieve minimum variance control. Linear quadratic Gaussian (LQG) control is the optimal control in theory. Its advantage is that the optimal gain coefficients of all control channels can be calculated through a single matrix operation, but the premise is the accurate statistical modeling of the controlled target. Considering the need for dynamic adjustment of the control algorithm, the processor must also be flexible so that it can quickly adapt to different control requirements. Although FPGA and DSP can achieve low latency, they have a long development cycle and are difficult to debug, and cannot meet the flexibility requirements; CPU and GPU algorithms are easy to modify, but their real-time performance is not as good as FPGA and DSP, especially the image data transmission delay during the acquisition process. The former needs to wait for the complete image frame to be transmitted before triggering the slope calculation, while the latter only needs to wait for a sub-aperture line to be transmitted before triggering the slope calculation. Summary of the invention
[0004] In order to solve the above technical problems, the present invention provides a computing and processing platform that takes into account the real-time and flexibility of the AO system. FPGA completes the slope extraction task with large calculation amount and fixed algorithm; multi-core CPU completes the voltage restoration and control operation with small calculation amount and frequent algorithm changes. The Hartmann image sub-aperture row transfer is used to reduce the delay in the transmission of the entire frame image data. Combined with the characteristics of flexible coding and convenient logic control of multi-core CPU, the goal of simultaneously meeting the real-time and flexibility is achieved, which is of great significance to improving the performance of the solar AO system.
[0005] To achieve the above purpose, the technical solution adopted by the present invention is as follows:
[0006] A computing and processing platform that takes into account the real-time and flexibility of the AO system, including: divided into an FPGA slope extraction module, a multi-core CPU voltage recovery module and a host computer status monitoring module according to the real-time computing task requirements, wherein:
[0007] The FPGA slope extraction module communicates unidirectionally with the multi-core CPU voltage recovery module via optical fiber, and the host computer status monitoring module communicates bidirectionally with the FPGA slope extraction module and the multi-core CPU voltage recovery module via a network;
[0008] The FPGA slope extraction module is used to receive the Hartmann camera signal, preprocess and extract the slope of the image data included in the camera signal, and send the calculated slope data to the multi-core CPU voltage recovery module;
[0009] The multi-core CPU voltage restoration module is used to receive slope data in real time, complete the restoration operation and logic control from slope data to voltage data, and output a control signal to the high-voltage amplifier;
[0010] The host computer status monitoring module is used to complete the UI interface display, status monitoring, human-computer interaction of the adaptive optical system, and parameter loading of the FPGA slope extraction module and the multi-core CPU voltage recovery module.
[0011] Furthermore, the image captured by the Hartmann camera is transmitted to the FPGA slope extraction module via a universal high-speed image transmission protocol; the FPGA slope extraction module is configured to adopt a sub-aperture row pipeline processing method for the image, mask the Hartmann image frame transfer time, and transmit the calculation result to the data acquisition card in the multi-core CPU voltage recovery module via optical fiber.
[0012] Furthermore, the FPGA slope extraction module selects the centroid algorithm or the normalized cross-correlation algorithm for slope calculation. For the normalized cross-correlation algorithm, when a sub-aperture row pixel completes transmission, the FPGA slope extraction module immediately starts slope calculation; for the centroid algorithm, the sub-aperture row pipeline processing mode is converted to a single-pixel pipeline processing mode, that is, when a pixel completes transmission, the FPGA slope extraction module immediately starts slope calculation.
[0013] Furthermore, the calculation result transmitted by the FPGA slope extraction module to the multi-core CPU voltage recovery module is a slope vector, and the transmission method is a pipeline processing method, and the transmission starts when the slope calculation of a sub-aperture row is completed.
[0014] Furthermore, after receiving the slope data, the multi-core CPU voltage restoration module completes the validity judgment of the slope data and the restoration operation of the slope data to the control voltage, and realizes the logic change and parameter adjustment of the predictive control algorithm.
[0015] Furthermore, the multi-core CPU server adopts the Linux operating system, transforms the non-real-time system into a hard real-time system by kernel trimming and building a real-time kernel patch, and realizes the parallelization of voltage restoration and wavefront control operations by core separation and thread binding.
[0016] Furthermore, the host computer status monitoring module sends the slope extraction related configuration file to the FPGA slope extraction module, and sends the voltage restoration related configuration file to the multi-core CPU voltage restoration module; the FPGA slope extraction module sends the original Hartmann image frame to the host computer status monitoring module, and the multi-core CPU voltage restoration module sends the real-time slope data and the restored voltage frame to the host computer status monitoring module.
[0017] The beneficial effects of the present invention are:
[0018] (1) The present invention adopts an FPGA architecture with strong timing control capability and low latency, and pre-processes and extracts slopes from images captured by the camera in a sub-aperture pipeline processing method. Compared with multi-core CPU and GPU architectures, the latter need to wait for the entire frame of the image to be transmitted before starting slope calculation. Data transmission and subsequent operations are a serial working mode, and the data transmission delay of the entire frame of Hartmann image is relatively long, which seriously affects the real-time closed-loop performance of the AO system. Customized FPGA has excellent performance and low latency characteristics for specific tasks. At the same time, the pipeline processing method can realize parallel work of data transmission and slope extraction, effectively reducing data transmission delay and ensuring the real-time performance of the architecture.
[0019] (2) The present invention uses a multi-core CPU with relatively low real-time performance but flexible coding to complete voltage restoration operations and logic control. Compared with FPGA and DSP architectures, the latter's hardware design and programming usually require higher professional knowledge and are suitable for specific computing tasks with fixed processing flows. It is difficult to achieve dynamic parameter changes and optimization operations for complex control logic. Multi-core CPUs have good versatility. During the application process, control parameters and strategies can be quickly and conveniently adjusted in combination with actual observation conditions to achieve the best correction effect, ensuring the flexibility of the architecture.
[0020] (3) The amount of voltage restoration calculation is relatively small, and the multi-core CPU resource consumption is low. Therefore, the idle cores of the multi-core CPU can be used for auxiliary functions such as performance evaluation of adaptive optical systems and atmospheric parameter estimation, providing sufficient scalability for the architecture.
[0021] The above advantages of the present invention provide a processing platform with both real-time performance and flexibility for the AO system, which is of great significance for improving the effective bandwidth of the solar AO system. BRIEF DESCRIPTION OF THE DRAWINGS
[0022] Figure 1 It is a module composition and workflow diagram of a computing and processing platform that takes into account the real-time and flexibility of the AO system;
[0023] Figure 2It is the sub-aperture arrangement diagram and driver arrangement diagram of the 1.8m solar AO system;
[0024] Figure 3 Completed the delay test results of 100,000 frames of image slope extraction for the FPGA slope extraction module;
[0025] Figure 4 Completed the delay test results of 100,000-frame image voltage restoration for the multi-core CPU voltage restoration module;
[0026] Figure 5 It is a flow chart of the pipeline processing of the present invention;
[0027] Figure 6 A timing diagram is calculated for the present invention. DETAILED DESCRIPTION
[0028] The present invention will be further described below in conjunction with the accompanying drawings and examples.
[0029] The invention provides a computing and processing platform which takes into account both the real-time performance and flexibility of an AO system. Firstly, a processor architecture is divided into an FPGA slope extraction module, a multi-core CPU voltage recovery module and a host computer state monitoring module according to the requirements of real-time computing tasks. The FPGA slope extraction module receives a Hartmann camera signal and performs preprocessing, then performs slope extraction, and sends the calculated slope data to the multi-core CPU voltage recovery module; the multi-core CPU module receives the slope data in real time, completes voltage recovery and control operations, and outputs a control signal to a correction device; the host computer state monitoring module completes state monitoring and human-computer interaction of the AO system, and loads parameters of the FPGA slope extraction module and the multi-core CPU voltage recovery module.
[0030] Among them, the slope calculation of the FPGA slope extraction module adopts the normalized cross-correlation algorithm, and the algorithm expression is as follows:
[0031]
[0032] in, Represents the coordinates of the upper left corner of the current real-time image. The cross-correlation factor value of the matching area, Represents the pixel coordinate position, Indicates the width and height of the matching area. Indicates the grayscale value of the pixels involved in the correlation operation in the current matching area. Represents the grayscale value of the pixel in the reference image that participates in the correlation operation. The purpose of this algorithm is to find a set of positions make Maximum, and finally the result is interpolated by quadratic parabola to obtain the wavefront slope with sub-pixel accuracy. Because the slope extraction algorithm is fixed and the amount of calculation is large, it is handed over to the FPGA with strong timing control capability and low latency to run; the voltage restoration operation needs to select the control algorithm and modify the control parameters according to the actual observation conditions to obtain the best correction effect, so it is handed over to the multi-core CPU with relatively low real-time performance but flexible coding. The above division enables the architecture to simultaneously meet the real-time and flexibility required by the solar adaptive optical system. Without loss of generality, the FPGA slope extraction module can flexibly select the centroid algorithm or the normalized cross-correlation algorithm for slope calculation according to specific application requirements. For the normalized cross-correlation algorithm, when a sub-aperture row pixel completes the transmission, the FPGA slope extraction module immediately starts the slope calculation; for the centroid algorithm, the sub-aperture row pipeline processing mode is converted to a single-pixel pipeline processing mode, that is, when a pixel completes the transmission, the FPGA slope extraction module immediately starts the slope calculation.
[0033] The FPGA slope extraction module uses a sub-aperture pipeline processing method for image reading, which can cover up the Hartmann full-frame image transfer time. The calculated slope vector is also transmitted to the data acquisition card in the multi-core CPU voltage recovery module through optical fiber in a pipeline processing method. On the one hand, the amount of slope vector data is much smaller than a complete frame of image, and on the other hand, the pipeline processing transmission method can cover up the slope transmission time, so the transmission delay between modules can be ignored.
[0034] The multi-core CPU voltage recovery module uses the Linux operating system, and transforms the non-real-time system into a hard real-time system by kernel trimming and building real-time kernel patches to reduce time jitter. Through core separation and thread binding, the voltage recovery and wavefront control operations are parallelized to improve the calculation speed. After receiving the slope data, the multi-core CPU voltage recovery module completes the validity judgment of the slope data and the recovery operation of the slope data to the control voltage, and can quickly realize the logic change and parameter adjustment of the predictive control algorithm.
[0035] The host computer status monitoring module sends the slope extraction related configuration file to the FPGA slope extraction module, and sends the voltage restoration related configuration file to the multi-core CPU voltage restoration module; the FPGA slope extraction module extracts frames of the original Hartmann image and sends them to the host computer status monitoring module, and the multi-core CPU voltage restoration module sends real-time slope data and restored voltage to the host computer status monitoring module.
[0036] Specifically, Figure 1As shown, a computing and processing platform that takes into account both the real-time performance and flexibility of the AO system includes an FPGA slope extraction module, a multi-core CPU voltage recovery module, and a host computer state monitoring module. The FPGA slope extraction module communicates unidirectionally with the multi-core CPU voltage recovery module through optical fiber, and the host computer state monitoring module communicates bidirectionally with the FPGA slope extraction module and the multi-core CPU voltage recovery module through the network. The FPGA slope extraction module is used to receive the Hartmann camera signal, pre-process and extract the slope of the image data, and send the calculated slope vector to the data acquisition card in the multi-core CPU voltage recovery module; the multi-core CPU voltage recovery module is used to receive the slope vector in real time, complete the voltage recovery operation and logic control, and output the control signal through the data acquisition card; the host computer state monitoring module is used to complete the UI interface display, state monitoring, human-computer interaction of the adaptive optical system, and parameter loading of the FPGA slope extraction module and the multi-core CPU voltage recovery module; the control signal is output to the acquisition card of the high-voltage amplifier and sent to the deformable mirror after digital-to-analog conversion.
[0037] like Figure 2 As shown in the figure, on the left is the 25×25 Hartmann microlens array sub-aperture arrangement diagram of the 1.8m solar AO system, with a total of 396 sub-apertures; on the right is the arrangement diagram of the deformable mirror driver, with a total of 451 driver units. Taking the solar AO system as an example, the image resolution collected by the system is 544 At 520 pixels, the camera transmission bandwidth is 16GB / s, and the transmission time of the entire image frame is at least 132us, while the transmission time of the sub-aperture row is only 1 / 25 of the transmission time of the entire frame.
[0038] like Figure 3 The figure shows the delay test result of the FPGA slope extraction module completing the slope extraction of 100,000 frames of images. The FPGA model is Xilinx ultra-large-scale VU9P FPGA, and the test image resolution is 544 520 pixels, including 396 sub-apertures, each with an effective resolution of 16 16 pixels. The slope calculation adopts the normalized cross-correlation algorithm based on FFT. The calculation delay of each frame starts from the triggering of the sub-aperture line interruption to the completion of the slope calculation of the last sub-aperture. The average delay is 157.29us, the minimum delay is 150.00us, and the peak-to-peak delay is 14.55us.
[0039] like Figure 4The figure shows the delay test results of the multi-core CPU voltage restoration module completing the voltage restoration of 100,000 frames of images. The multi-core CPU model is Intel's 14th-generation Core i9, the input slope vector length is 792, and the number of drive units is 451. The control algorithm uses PI control. The calculation delay of each frame starts from the interrupt trigger to the completion of all voltage signal calculations. The average delay is 20.31us, the minimum delay is 16.00us, and the peak-to-peak delay is 144.80us.
[0040] like Figure 5 The figure shows the pipeline processing flow chart of the present invention. When a sub-aperture row of the camera image is transmitted, the slope calculation can be triggered. The 1.8m solar AO system has 25 sub-aperture rows. The time consumed by using the sub-aperture row pipeline transmission method is 1 / 25 of the time consumed by the complete frame transmission of the image, which greatly reduces the closed-loop delay of the AO system.
[0041] like Figure 6 The figure shows a schematic diagram of the calculation timing of the present invention. The FPGA slope extraction module receives the sub-aperture row image transmitted by the camera through the general high-speed image transmission protocol and starts preprocessing and slope calculation. After the slope calculation of a sub-aperture is completed, it is sent to the multi-core CPU voltage recovery module through the optical fiber, so that the image frame transfer time and the slope transmission time can be concealed; after the slope vector is transmitted, the multi-core CPU voltage recovery module performs voltage recovery operation and logic control, and outputs a control signal to the high-voltage amplifier.
[0042] The specific embodiments described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.
Claims
1. A computing and processing platform that takes into account both the real-time performance and flexibility of the AO system, characterized in that: include: According to the real-time computing task requirements, it is divided into FPGA slope extraction module, multi-core CPU voltage recovery module and host computer status monitoring module, among which, The FPGA slope extraction module communicates unidirectionally with the multi-core CPU voltage recovery module via optical fiber, and the host computer status monitoring module communicates bidirectionally with the FPGA slope extraction module and the multi-core CPU voltage recovery module via a network; The FPGA slope extraction module is used to receive the Hartmann camera signal, preprocess and extract the slope of the image data included in the camera signal, and send the calculated slope data to the multi-core CPU voltage recovery module; The multi-core CPU voltage restoration module is used to receive slope data in real time, complete the restoration operation and logic control from slope data to voltage data, and output a control signal to the high-voltage amplifier; The host computer status monitoring module is used to complete the UI interface display, status monitoring, human-computer interaction of the adaptive optical system, and parameter loading of the FPGA slope extraction module and the multi-core CPU voltage recovery module.
2. A computing and processing platform that takes into account both the real-time performance and flexibility of the AO system as described in claim 1, characterized in that: The image captured by the Hartmann camera is transmitted to the FPGA slope extraction module via a universal high-speed image transmission protocol; the FPGA slope extraction module is configured to adopt a sub-aperture row pipeline processing method for the image, mask the Hartmann image frame transfer time, and transmit the calculation result to the data acquisition card in the multi-core CPU voltage recovery module via optical fiber.
3. A computing and processing platform that takes into account both the real-time performance and flexibility of the AO system as described in claim 2, characterized in that: The FPGA slope extraction module selects the centroid algorithm or the normalized cross-correlation algorithm for slope calculation. For the normalized cross-correlation algorithm, when a sub-aperture row pixel completes transmission, the FPGA slope extraction module immediately starts slope calculation; for the centroid algorithm, the sub-aperture row pipeline processing mode is converted to a single-pixel pipeline processing mode, that is, when a pixel completes transmission, the FPGA slope extraction module immediately starts slope calculation.
4. A computing and processing platform that takes into account both the real-time performance and flexibility of the AO system as described in claim 2, characterized in that: The calculation result transmitted by the FPGA slope extraction module to the multi-core CPU voltage restoration module is a slope vector, and the transmission method is a pipeline processing method. The transmission starts when the slope calculation of a sub-aperture row is completed.
5. A computing and processing platform that takes into account both the real-time performance and flexibility of the AO system as described in claim 1, characterized in that: After receiving the slope data, the multi-core CPU voltage restoration module completes the validity judgment of the slope data and the restoration operation of the slope data to the control voltage, and realizes the logic change and parameter adjustment of the predictive control algorithm.
6. A computing and processing platform that takes into account both the real-time performance and flexibility of the AO system as described in claim 1, characterized in that: The multi-core CPU server adopts the Linux operating system, transforms the non-real-time system into a hard real-time system by kernel cutting and building real-time kernel patches, and realizes parallelization of voltage restoration and wavefront control operations by core separation and thread binding.
7. A computing and processing platform that takes into account both the real-time performance and flexibility of the AO system as described in claim 1, characterized in that: The host computer status monitoring module sends the slope extraction related configuration file to the FPGA slope extraction module, and sends the voltage restoration related configuration file to the multi-core CPU voltage restoration module; the FPGA slope extraction module sends the original Hartmann image frame to the host computer status monitoring module, and the multi-core CPU voltage restoration module sends the real-time slope data and the restored voltage frame to the host computer status monitoring module.