Adaptive optical matrix calculation acceleration method based on ARM architecture

Abstract

The invention relates to a self-adaptive optical matrix calculation acceleration method based on an ARM framework, and belongs to the field of self-adaptive optical wavefront correction technology and embedded parallel calculation. The method comprises the following steps: analyzing a mathematical mapping relationship between a microlens array and wavefront phase distribution in a Hartmann sensor image, and constructing a wavefront matrix operation model based on an ARM multi-core architecture; designing a Hartmann image data rearrangement strategy, dividing a single-frame Hartmann image into multiple task calculation units, and realizing parallelization processing of wavefront slope calculation; a wavefront matrix operation kernel is optimized according to ARM NEON instruction set characteristics, and vectorization calculation of multi-channel wavefront reconstruction is realized by adopting an SIMD parallelization technology; and computing resource allocation is coordinated through a dynamic thread scheduling algorithm, and the task allocation proportion among multiple cores is adjusted according to the real-time load condition. According to the method, the wavefront calculation efficiency is remarkably improved, so that the overall bandwidth of the adaptive optical system is greatly improved.

Description

Technical Field

[0001] The present invention belongs to the field of adaptive optical wavefront correction technology and embedded parallel computing, and specifically relates to an adaptive optical matrix calculation acceleration method based on ARM architecture. The method is particularly suitable for embedded adaptive optical systems that achieve high real-time wavefront reconstruction through multi-threaded optimization and instruction set acceleration. Background Art

[0002] In adaptive optics systems, the wavefront processor serves as the core processing unit for real-time wavefront correction. Its computing performance directly determines the system bandwidth and correction accuracy. Traditional solutions often use heterogeneous computing architectures such as DSP arrays, FPGAs, or GPUs. While these solutions can meet high-speed computing requirements, they suffer from high power consumption, high costs, and low system integration.

[0003] In recent years, ARM architecture processors have shown great potential in the field of embedded real-time computing due to their excellent energy efficiency and scalability, but dedicated ARM architecture solutions for wavefront processing are still in the exploratory stage. Current public research shows that ARM-based wavefront processors have the following key problems: (1) Existing solutions often directly transplant traditional algorithms without deep optimization for the parallel characteristics of ARM multi-core architecture, resulting in low thread scheduling efficiency; (2) The vectorization acceleration capabilities of ARM instruction sets (such as NEON SIMD) are underdeveloped, and wavefront matrix operations still rely on serial computing mode. The above technical bottlenecks have seriously restricted the practical application of ARM architecture in the field of adaptive optics, and a breakthrough is urgently needed through algorithm-architecture collaborative optimization. Summary of the Invention

[0004] This paper addresses the low matrix computation efficiency and insufficient parallel resource utilization of existing ARM-based wavefront processors. By proposing an ARM-based adaptive optical matrix computation acceleration method, the paper addresses the issues of low matrix computation efficiency and insufficient parallel resource utilization in existing ARM-based wavefront processors. Its core innovation lies in the deep collaborative optimization of algorithms and hardware characteristics, overcoming the performance bottlenecks of traditional wavefront processors and significantly improving processor computing speed and system bandwidth.

[0005] To achieve the above object, the technical solution adopted by the present invention is as follows:

[0006] An adaptive optical matrix calculation acceleration method based on ARM architecture, comprising:

[0007] Step S110: Analyze the mathematical mapping relationship between the microlens array and the wavefront phase distribution in the Hartmann sensor image, and construct a wavefront matrix operation model based on the ARM multi-core architecture;

[0008] Step S120: designing a Hartmann image data rearrangement strategy to divide a single-frame Hartmann image into multi-task computing units to achieve parallel processing of wavefront slope calculation;

[0009] Step S130: Optimizing the wavefront matrix operation kernel according to the characteristics of the ARM NEON instruction set, and using SIMD parallelization technology to implement vectorized calculation of multi-channel wavefront reconstruction;

[0010] Step S140: Coordinate computing resource allocation through a dynamic thread scheduling algorithm and adjust the task allocation ratio among multiple cores according to real-time load conditions.

[0011] Beneficial effects

[0012] This paper leverages the multi-threaded processing capabilities and high-speed instruction set optimization of the ARM architecture, combined with the characteristics of wavefront matrix operations in adaptive optics. It proposes a multi-task parallel computation method for rearranging a single Hartmann image, along with a multi-channel data parallel operation strategy based on instruction set optimization. This method significantly improves the efficiency of wavefront computation, thereby significantly increasing the overall bandwidth of the adaptive optics system.

[0013] Compared with the prior art, the present invention has the following significant advantages:

[0014] Breakthrough improvement in computing speed

[0015] On the Rockchip RK3588 platform, the wavefront calculation time for a 256×256 camera target surface is reduced from 220us in traditional solutions to 60us, and the real-time processing rate is increased by about 3.6 times;

[0016] NEON instruction optimization improves matrix multiplication and addition performance from 8.9 GFLOPS to 42.0 GFLOPS, an increase of approximately 4 times compared to the unoptimized implementation.

[0017] Leading system bandwidth indicators

[0018] Through multi-core load balancing technology, the center of mass coordinate calculation latency was reduced from 72.89μs to 8.36μs in quad-core full-speed mode, which is approximately 88.5% shorter than the traditional thread scheduling solution on the same platform.

[0019] The system's single-frame image calculation delay is controlled within 60μs, achieving 2000Hz Hartmann camera closed-loop correction and meeting the high-frequency correction requirements under strong turbulence conditions.

[0020] ARM architecture-specific advantages

[0021] The energy efficiency ratio reaches 22.4GFLOPS / W, which is 76% lower than the power consumption of the X86 solution with the same performance;

[0022] The hardware cost is only 1 / 5 of the FPGA solution, and it supports rapid algorithm iteration through the standard Linux kernel. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 Schematic flow chart of an adaptive optical matrix calculation acceleration method based on ARM architecture according to the present invention;

[0024] Figure 2 is the geometric mapping relationship between the microlens array and the wavefront phase distribution in the Hartmann sensor image;

[0025] Figure 3 A multithreaded decomposition method for matrix-vector multiplication;

[0026] Figure 4 ARM NEON instruction set and loop traversal data processing method;

[0027] Figure 5 Comparison of computing performance between ARM NEON instruction set and loop traversal;

[0028] Figure 6 Modeling rearrangement methods for Hartmann image reconstruction;

[0029] Figure 7 This is the far-field beam closed-loop correction effect after using the ARM-based matrix acceleration method;

[0030] Figure 8 The figure shows the effect of closed-loop correction of 2000Hz Hartmann camera. DETAILED DESCRIPTION

[0031] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments described herein. Rather, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0032] Figure 1 Schematic flow chart of the adaptive optical matrix calculation acceleration method based on ARM architecture according to the present invention. Figure 1 Describe the execution process of this method. Figure 1 As shown, the method includes:

[0033] Step S110: Analyze the mathematical mapping relationship between the microlens array and the wavefront phase distribution in the Hartmann sensor image, and construct a wavefront matrix operation model based on the ARM multi-core architecture, so as to model the Hartmann image reconstruction process as a parallelizable matrix multiplication operation sequence.

[0034] Figure 2is the geometric mapping relationship between the microlens array and the wavefront phase distribution in the Hartmann sensor image, where the distorted wavefront refers to the wavefront whose wavefront shape deviates from the ideal plane or sphere, the undistorted wavefront refers to the wavefront under ideal conditions, and the sub-spot imaging refers to the imaging on the detector after each microlens (or sub-aperture) focuses the wavefront within its coverage range. The position of each sub-spot is related to the degree of distortion of the wavefront. The more skewed the wavefront, the more the sub-spot deviates from the ideal center position. According to the principles of geometric optics, the gradient of the wavefront phase is the slope of the wavefront. Assuming the wavefront phase is , in the two-dimensional case, the slope of the wavefront phase exist and The directions are:

[0035] (1)

[0036] When the incident light beam is distorted, the position and shape of the sub-spot will change accordingly. , The offset in the two directions can be used to calculate the average slope of the corresponding wavefront in the two directions within each sub-aperture range:

[0037] (2)

[0038] (3)

[0039] in, It's a pixel The received light intensity signal, It's a pixel 's coordinates. is the coordinate of the sub-spot center of mass, is the average slope of the wavefront in two directions, is the focal length of the microlens.

[0040] In actual wavefront correction, the direct slope method is usually used to take the control voltage of each driver as the correction control target. By analyzing the effect of unit voltage applied to each driver on the corresponding subaperture slope, a slope response matrix between driver voltage and slope is established. , the correction process of the distortion phase can be expressed by the matrix:

[0041] (4)

[0042] in, is the driver voltage, is the slope of the wavefront phase.

[0043] For an adaptive optical system with m sub-apertures and n actuators, the expanded matrix is ​​expressed as:

[0044] (5)

[0045] Among them, the left column of the equal sign is the slope matrix of the wavefront phase of the m sub-apertures, and the first matrix on the right side of the equal sign is the slope response matrix , among which - is the influence function of the 1st to nth drivers on the 1st to mth sub-apertures in the x-direction and y-direction, and the second column to the right of the equal sign is the driver voltage matrix of the n drivers.

[0046] Slope response matrix It can be obtained through theoretical calculation or experiment. For the wavefront slope including phase distortion , through the generalized inverse operation, the control voltage that minimizes the residual slope and the control energy can be obtained as:

[0047] (6)

[0048] Therefore, the calculation and control process of the wavefront phase can be simplified as follows:

[0049] The light intensity image received from the Hartmann sensor is represented in matrix form as follows for an image with a sub-aperture size of N×N pixels and an arrangement of p×p:

[0050] (7)

[0051] (8)

[0052] in, is the image matrix of a single sub-aperture; is the grayscale value of each pixel, is the light intensity image.

[0053] 1. Find the centroid position of each subaperture and obtain the average slope.

[0054] 2. Based on the above-mentioned Hartmann wavefront sensor principle, calculate the average slope of all sub-apertures and arrange them in matrix form to obtain the average slope vector of the entire image .

[0055] 3. Pre-store the generalized inverse of the slope response matrix , once the slope vector After the calculation is completed, the control voltage V can be calculated according to formula (6).

[0056] Step S120: designing a Hartmann image data rearrangement strategy, dividing a single-frame Hartmann image into multi-task computing units, and realizing parallel processing of wavefront slope calculation.

[0057] Designing a Hartmann image data rearrangement strategy may include: adopting a sub-aperture array division method to rearrange the spot data of adjacent microlens arrays in the Hartmann image according to the memory alignment requirements of the computing unit, so that the cache line loaded by each ARM computing core contains the complete data required for sub-aperture slope calculation.

[0058] Specifically, it can include: data rearrangement and parallel processing, designing a Hartmann image data rearrangement strategy based on cache line optimization, and specific implementation methods include:

[0059] (a) Divide a single-frame Hartmann image into p×p sub-aperture calculation units according to the microlens array arrangement, where p = intensity image width Subaperture size.

[0060] (b) Using the row-column cross partitioning strategy, the spot data of adjacent sub-apertures are rearranged in a 64-byte alignment manner.

[0061] (c) Allocate a continuous memory block to each ARM computing core to ensure that a single cache load can complete the slope calculation of one sub-aperture.

[0062] like Figure 3 As shown in the figure, the dot product of each row of the matrix-vector multiplication and the vector is a relatively independent calculation task. By utilizing the multi-core and multi-threading features of ARM, each thread can be assigned to process the dot product of different rows of the matrix and the vector, thereby significantly improving the calculation speed. Figure 4 As shown in the figure, after implementing effective core isolation on the quad-core Cortex-A76 processor, the computing performance of the quad-core processor reaches its peak when the computing amount is greater than 0.4 MFLOPs, reaching 41.3 GFLOPS, which is about 4 times the performance of a single core. Figure 5 As shown, the Hartmann image is based on a sub-aperture of N×N pixels, which is rearranged into 1×N 2 A one-dimensional matrix. If the image pixel width is not an integer number of bytes, pad it with blank bits (0) to align it. If the image row width is not a multiple of 64 bytes, pad it with blank bytes to align it to facilitate instruction set optimization and multi-threaded parallel computing.

[0063] Step S130 , optimizing the wavefront matrix operation kernel according to the characteristics of the ARM NEON instruction set, and using SIMD parallelization technology to implement vectorized calculation of multi-channel wavefront reconstruction.

[0064] The wavefront reconstruction includes wavefront slope calculation and restoration as well as voltage calculation, all of which are performed in the form of matrix multiplication. The rearrangement strategy mentioned in step S120 is only for slope calculation.

[0065] The wavefront matrix operation kernel optimized for the ARM NEON instruction set includes:

[0066] (a) Decomposing the matrix operation data of wavefront reconstruction into four 32-bit block processing units;

[0067] (b) Use register interleaving access strategy to eliminate pipeline stalls caused by data dependencies.

[0068] Specifically, this step may also include:

[0069] (a) Decomposing the wavefront reconstruction matrix into 4×4 floating-point matrix block processing units to match the 128-bit bit width of the NEON register.

[0070] (b) Design a register rotation multiplexing mechanism and implement multiplication and addition pipeline optimization through the VLD1.32 / VMLA.F32 instruction combination.

[0071] (c) Use register interleaving access strategy to eliminate pipeline stalls caused by data dependencies.

[0072] like Figure 6 As shown in Figure 1, Neon instructions can simultaneously process four 32-bit floating-point values ​​in a single operation, significantly reducing loop execution times, typically by a factor of four. The Cortex-A76 processor provides 32 Neon registers, divided into four groups for processing different rows of a matrix, or sub-apertures. Each row of the matrix is ​​loaded into a different group of registers, with no data dependencies between registers in each group. This eliminates pipeline stalls caused by data dependencies by alternating between register loading and operations. Figure 7 The performance comparison between loop traversal and Neon instructions is shown. The results show that when the vector length is moderate, Neon instructions can improve the computing performance from 8.9 GFLOPS to 42.0 GFLOPS, an improvement of about 4 times.

[0073] Step S140 , coordinating computing resource allocation through a dynamic thread scheduling algorithm, and adjusting the task allocation ratio among multiple cores according to real-time load conditions.

[0074] Among them, the dynamic thread scheduling algorithm adopts the core idea of ​​combining dynamic and static:

[0075] (a) Static binding: For tasks with known high computational requirements, large cores are pre-bound to ensure critical performance.

[0076] (b) Dynamic adjustment: Through real-time monitoring and load migration, it can flexibly respond to sudden loads and prevent resource overload.

[0077] The process of coordinating computing resource allocation through a dynamic thread scheduling algorithm includes:

[0078] (a) High-computation thread binding: During the initialization phase of the wavefront reconstruction task, the matrix operation thread is forcibly bound to the large-core cluster (CPU4-CPU7) through the pthread_setaffinity_np() interface to ensure that it has exclusive large-core computing resources;

[0079] (b) After the rearrangement and optimization, resource monitoring and thread scheduling are performed on the system. Through the embedded platform's real-time performance monitoring interface, CPU utilization, memory bandwidth usage, and temperature metrics of each computing core are periodically collected. The monitoring period can be dynamically adjusted to 10-50ms. When the load of the large core (Cortex-A76) exceeds a predetermined threshold for three consecutive cycles, a load migration warning is triggered, and a load balancing instruction is sent to the scheduler through the kernel event notification mechanism. The predetermined threshold can be 80%.

[0080] Dynamic resource scheduling control involves building a load-aware dynamic thread scheduling system, including:

[0081] (a) System-level resource monitoring: Through the real-time performance monitoring interface of the embedded system, the CPU utilization and memory load indicators of each computing core are periodically collected;

[0082] (b) Dynamic Thread Scheduling Strategy: A dynamic thread affinity scheduling strategy based on real-time load feedback is designed to achieve inter-core load balancing through the following mechanisms: 1. A thread-core mapping table is established to dynamically adjust thread binding status based on the computational load of each thread and the core load ratio. 2. Wavefront reconstruction tasks are prioritized based on computational load estimation, prioritizing matrix operation threads to the large cores (Cortex-A76) and data preprocessing threads to the small cores (Cortex-A55).

[0083] The method described above can be implemented on the Rockchip RK3588 embedded platform. The wavefront detector and wavefront corrector are connected via the PCIe bus to form a closed-loop control system, meeting the real-time wavefront processing rate requirement of more than 2000 Hz.

[0084] Required characteristics for the implementation of a systems approach:

[0085] Deployment platform: Built on an embedded processing unit based on the Rockchip RK3588 chip (quad-core Cortex-A76 + quad-core Cortex-A55 architecture).

[0086] Data transmission: The sCMOS wavefront detector and MEMS deformable mirror controller are connected via a PCIe 3.0 x4 interface, with end-to-end latency less than 300 μs.

[0087] Real-time performance: A single-frame calculation delay of 60μs and a stable closed-loop processing rate of 2000Hz are achieved under a 256×256 camera target configuration.

[0088] The technical principle of this invention is based on the deep coupling of the hardware characteristics of the ARM architecture and the algorithmic characteristics of wavefront matrix operations, achieving computational acceleration through three-level collaborative optimization:

[0089] Architecture-level optimization: Leveraging the distributed computing capabilities of ARM multi-core processors, the traditional serial wavefront reconstruction process is decoupled into subtask streams that can be executed in parallel, and the inter-core communication overhead is reduced through a NUMA (Non-Uniform Memory Access)-aware data distribution strategy.

[0090] Instruction-level acceleration: A 4×4 floating-point matrix block operation mode is designed for the 128-bit vector processing unit of the ARM NEON SIMD instruction set. A single instruction can simultaneously complete the multiplication and addition operations of four pairs of floating-point numbers, increasing the theoretical computing throughput to four times that of scalar operations.

[0091] Data flow reconstruction: Based on the cache line characteristics of the ARM architecture, the Hartmann image data is reorganized into 64-byte aligned data, eliminating the performance loss caused by non-aligned and non-contiguous memory access, and improving data loading efficiency by more than 35%.

[0092] After the matrix operation is optimized, the system single-frame image calculation delay is controlled at about 60μs. The delay is shown in Table 1 below, and the 2000Hz Hartmann camera closed-loop correction is achieved. The effect is as follows Figure 8 As shown, it meets the high-frequency correction requirements under strong turbulence conditions.

[0093] Table 1

[0094] In the description provided herein, a large number of specific details are described. However, it is understood that embodiments of the present invention can be practiced without these specific details. In some instances, well-known methods, structures, and techniques are not shown in detail so as not to obscure the understanding of this description.

[0095] Although the present invention has been described with respect to a limited number of embodiments, those skilled in the art, having benefit of the foregoing description, will appreciate that other embodiments are contemplated within the scope of the invention thus described. Furthermore, it should be noted that the language used in this specification has been selected primarily for readability and instructional purposes, and not for the purpose of explaining or limiting the subject matter of the present invention.

Claims

1. An adaptive optical matrix calculation acceleration method based on ARM architecture, characterized in that: include: Step S110: Analyze the mathematical mapping relationship between the microlens array and the wavefront phase distribution in the Hartmann sensor image, and construct a wavefront matrix operation model based on the ARM multi-core architecture; Step S120: designing a Hartmann image data rearrangement strategy to divide a single-frame Hartmann image into multi-task computing units to achieve parallel processing of wavefront slope calculation; Step S130: Optimizing the wavefront matrix operation kernel according to the characteristics of the ARM NEON instruction set, and using SIMD parallelization technology to implement vectorized calculation of multi-channel wavefront reconstruction; Step S140: Coordinate computing resource allocation through a dynamic thread scheduling algorithm and adjust the task allocation ratio among multiple cores according to real-time load conditions.

2. The adaptive optical matrix calculation acceleration method based on ARM architecture according to claim 1, characterized in that: In step S120, the design of the Hartmann image data rearrangement strategy includes: using a sub-aperture row division method to rearrange the spot data of adjacent microlens arrays in the Hartmann image according to the memory alignment requirements of the computing unit, so that the cache line loaded by each ARM computing core contains the complete data required for sub-aperture slope calculation.

3. The adaptive optical matrix calculation acceleration method based on ARM architecture according to claim 1, characterized in that: Step S120 includes: (a) Divide a single-frame Hartmann image into p×p sub-aperture calculation units according to the microlens array arrangement, where p = Hartmann image width subaperture size; (b) Using the row-column cross partitioning strategy, the spot data of adjacent sub-apertures are rearranged in a 64-byte alignment manner; (c) Allocate a continuous memory block to each ARM computing core to ensure that a single cache load can complete the slope calculation of one sub-aperture.

4. The adaptive optical matrix calculation acceleration method based on ARM architecture according to claim 1, characterized in that: In step S130, optimizing the wavefront matrix operation kernel according to the ARM NEON instruction set characteristics includes: (a) Decomposing the matrix operation data of wavefront reconstruction into four 32-bit block processing units; (b) Use register interleaving access strategy to eliminate pipeline stalls caused by data dependencies.

5. The adaptive optical matrix calculation acceleration method based on ARM architecture according to claim 4, characterized in that: Step S130 includes: (a) Decomposing the wavefront reconstruction matrix into 4×4 floating-point matrix block processing units to match the 128-bit width of the NEON register; (b) Design a register rotation multiplexing mechanism and optimize the multiplication and addition pipeline through the VLD1.32 / VMLA.F32 instruction combination; (c) Use register interleaving access strategy to eliminate pipeline stalls caused by data dependencies.

6. The adaptive optical matrix calculation acceleration method based on ARM architecture according to claim 1, characterized in that: The dynamic thread scheduling algorithm in step S140 adopts the core idea of ​​combining dynamic and static methods: (a) Static binding: For tasks with known high computational requirements, large cores are pre-bound to ensure critical performance. (b) Dynamic adjustment: Through real-time monitoring and load migration, it can flexibly respond to sudden loads and prevent resource overload.

7. The adaptive optical matrix calculation acceleration method based on ARM architecture according to claim 6, characterized in that: In step S140, the process of coordinating computing resource allocation through the dynamic thread scheduling algorithm includes: (a) High-computation thread binding: During the initialization phase of the wavefront reconstruction task, the matrix operation thread is forcibly bound to the large-core cluster through the pthread_setaffinity_np() interface to ensure that it has exclusive large-core computing resources; (b) Perform resource monitoring and thread scheduling on the rearranged and optimized system. Periodically collect CPU utilization, memory bandwidth occupancy, and temperature indicators of each computing core through the real-time performance monitoring interface of the embedded platform. When it is detected that the load of a large core exceeds a predetermined value for three consecutive cycles, a load migration warning is triggered, and a load balancing instruction is sent to the scheduler through the kernel event notification mechanism.

8. The adaptive optical matrix calculation acceleration method based on ARM architecture according to claim 7, characterized in that: The predetermined value is 80%.

9. The adaptive optical matrix calculation acceleration method based on ARM architecture according to claim 7, characterized in that: The monitoring period of the real-time performance monitoring interface is dynamically adjusted to 10-50ms.

10. The adaptive optical matrix calculation acceleration method based on ARM architecture according to claim 1, characterized in that: The method is implemented on the Rockchip RK3588 embedded platform. A wavefront detector and a wavefront corrector are connected via a PCIe bus to form a closed-loop control system, meeting the real-time wavefront processing rate requirement of more than 2000 Hz.

Images