Image rapid calculation processing method based on wave aberration correction

Abstract

The invention relates to an image processing method, in particular to a rapid image calculation processing method based on wave aberration correction, and solves the problem that the existing scheme is low in speed and cannot meet the requirements of wave aberration correction and low power consumption and low time delay when a satellite platform is used at the same time. The method comprises the following steps of: 1, initializing parameters: 1.1) setting a wave aberration control voltage initial matrix; 1.2) setting a response function coefficient matrix # imgabs0 #, and splitting and storing the matrix # imgabs0 # to DDR3; 1.3) setting a voltage gain matrix; step 2, reading a wavefront image; 3, windowing processing and subpackaging are carried out; 4, a mass center offset matrix # imgabs1 # is calculated; step 5, calculating a product matrix # imgabs2; 6, calculating a current wave aberration control voltage matrix; 7, obtaining a wavefront image after adjusting the shape of the deformable mirror surface; 8, judging whether an ending instruction is received or not; if not, returning to the step 2; and if yes, completing. The method is suitable for image processing of Hartmann wavefront correction.

Description

Technical Field

[0001] This invention relates to an image processing method, and more particularly to a fast image calculation and processing method based on wavelet aberration correction. Background Technology

[0002] Image storage and retrieval refers to the process of converting received data into a matrix or other data stream according to rules, storing it in a medium, and then reading the data back for image processing. Commonly used storage media include NOR flash and DDR. Viewing image storage and retrieval as a pipeline, data format conversion is required between different storage media to complete data transfer.

[0003] With the increasing demands for space target detection, the real-time requirements for high-speed image storage and retrieval against space target backgrounds are also rising. The rapid development of integrated circuit technology provides hardware support for solving this problem; however, how to cleverly utilize advanced integrated circuit hardware resources to achieve high-speed, real-time image storage and retrieval has become a challenging task. In recent years, high-speed real-time sensor image processing systems based on FPGA and DDR technologies have received widespread attention. These systems can achieve high-speed image acquisition, preprocessing, and storage, providing powerful support for various applications.

[0004] The common storage solutions currently available include the following: (1) High-speed remote image data storage system based on LVDS. This system uses an FPGA chip as the control core, and combines low-voltage differential signal technology (LVDS) interface deserialization and driver chip to ensure effective reception of remote data; it uses an alternating dual-plane FLASH programming method to control the storage of image data. (2) In response to the demand for high-speed image data FPGA large-capacity storage, some scholars have proposed an invalid block management strategy based on a high-speed parallelized FLASH large-capacity data storage structure. (3) Some scholars have also proposed a custom image encoding method for approximate storage substrate. They found that storing encoded images in approximate memory indiscriminately will produce unacceptable and uncontrollable quality problems. (4) Some researchers have explored the problem of using DDR for image storage. (5) Some papers have also proposed a design method for a high-speed image data transmission system based on DDR2. The system designed by this method uses FPGA internal FIFO resources and off-chip DDR2 hierarchical caching mechanism to meet the high throughput requirements of images and the large capacity requirements of data caching. All the aforementioned storage solutions address the issue of high-speed image data storage and provide specific solutions. These solutions share the commonality of emphasizing high speed, large capacity, and reliability, and all utilize FPGAs as the control core of the storage system, leveraging their parallel processing capabilities and flexibility to achieve high-speed, large-capacity data storage. The differences lie in the specific technologies and methods employed, targeting different application needs and problems. The shortcomings of these solutions are: firstly, while storage and retrieval speeds have improved, they remain relatively slow, and real-time performance remains poor, failing to meet the real-time requirements of wavelet aberration correction; secondly, most are used in terrestrial equipment, requiring large-capacity GPU servers, while satellite platforms have limited hardware resources, limiting the capacity available for image storage, retrieval, and computation. Therefore, they cannot meet the low power consumption and low latency requirements of satellite platform applications, making direct application on satellite platforms difficult; even those that can be directly applied to satellite platforms suffer from slow speeds and poor real-time performance, failing to meet the requirements of wavelet aberration correction.

[0005] In addition, some scholars have proposed a storage scheme based on FPGA-controlled Nand Flash arrays to achieve high-speed pipelined storage. This scheme writes image data into the FLASH storage array via the CameraLink input communication interface and uses DMA transfer technology to upload the stored image file to the computer's hard drive for further processing. Simultaneously, it uses SDRAM video memory to refresh data in real time, and the FPGA constructs the corresponding VGA signal, ultimately achieving real-time image display at 100MB / s. However, this scheme only achieves real-time image display and does not perform any calculations or interpretations of the image data.

[0006] Some researchers have proposed studying image storage and retrieval based on DDR4-based real-time optoelectronic image storage technology. They first introduced key DDR4 technologies and working principles; then, in the interface module, they studied triple-buffer interface technology, enabling real-time data read / write functionality across multiple devices; furthermore, in the read and write operation modules, they proposed a periodic distributed refresh algorithm to solve the instruction lag problem. However, this approach only improves the initial response speed and does not accelerate subsequent storage, retrieval, or computation speeds.

[0007] In summary, developing a rapid image processing method capable of fast storage, retrieval, and computation, which can simultaneously meet the real-time requirements of wavelet aberration correction and the high reliability, low power consumption, and low latency requirements of storage, retrieval, and computation when used on satellite platforms, is an urgent problem to be solved.

[0008] Below are explanations of the English vocabulary involved: NOR flash: A type of non-volatile memory with fast read capabilities, typically used to store firmware and operating systems; DDR: Double Data Rate SDRAM, synchronous dynamic random access memory; DDR2: Double Data Rate 2 SDRAM, a second-generation synchronous dynamic random access memory with double data rate. DDR3: Double Data Rate 3 SDRAM, a third-generation synchronous dynamic random access memory with double data rate. DDR4: Double Data Rate 4 SDRAM, a fourth-generation synchronous dynamic random access memory with double data rate. FPGA: Field Programmable Gate Array; LVDS: Low Voltage Differential Signaling; FLASH: A type of non-volatile memory that can quickly read and write data; FIFO: First In First Out, a type of data buffer whose main characteristic is that data is written sequentially and then output in the same order, meaning that the data that goes in first is taken out first. GPU: Graphics Processing Unit; Nand Flash: NAND flash memory, a non-volatile memory technology mainly used for data storage; CameraLink: A serial communication protocol specifically designed for machine vision applications; DMA: Direct Memory Access, is a hardware feature that allows certain hardware subsystems to directly access the system's memory without going through the central processing unit (CPU). SDRAM: Synchronous Dynamic Random-Access Memory; VGA: Video Graphics Array, a standard for transmitting video using analog signals. Summary of the Invention

[0009] The purpose of this invention is to solve the technical problems of existing image storage and retrieval schemes, which have slow storage and retrieval speeds, poor real-time performance, and cannot simultaneously meet the real-time requirements of wavelet aberration correction as well as the requirements of high reliability, low power consumption, and low latency for storage, retrieval, and computation when used on satellite platforms. The invention provides a fast image computation and processing method based on wavelet aberration correction.

[0010] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows: A fast image processing method based on wavefront aberration correction, which is implemented based on a wavefront aberration correction system, is characterized by the following steps: Step 1: Initialize the wavelet aberration correction control system parameters, specifically including: 1.1) Based on the number of correction voltage loops N and the calibration results of the wave aberration correction system, set the initial matrix of the wave aberration control voltage in the NOR flash. The initial matrix of the wave aberration control voltage for Matrix; 1.2) Based on Hartmann's number of sub-apertures 'a' and the number of correction voltage loops 'N', set the response function coefficient matrix of centroid offset and wave aberration control voltage adjustment. The response function coefficient matrix The size is , Then, divide it into n groups, grouping each of columns C into a single group. The parallel coefficient matrix, the Next, in parallel, the coefficients with the same row number in each H column of each parallel coefficient matrix are concatenated and stored one by one into the corresponding storage unit in a row of DDR3. In DDR3, they are stored in the form of n groups, each group with Z rows and each row storing M coefficients; where 2≤H≤C; where Z is the value obtained by rounding up the quotient of C divided by H. 1.3) Based on the actual hardware environment of the wavelet aberration correction system, set the voltage gain matrix in the NOR flash. The voltage gain matrix The size is ; Step 2: Read the wavefront image; Step 3: Based on the number of sub-apertures 'a' of Hartmann, the read wavefront image is windowed and divided into packets to obtain b windowed image packets. Each windowed image packet contains a / b windowed images, where b is the set number of packets and a / b is an integer. Step 4: For each windowed image packet obtained in Step 3, calculate the centroid offset of each windowed image in the X and Y directions in parallel to obtain the centroid offset matrix. The centroid offset matrix The size is ; Step 5: The FPGA reads the response function coefficient matrix stored in step 1.2) from DDR3. In DDR3, each vertical group consists of a / b coefficients. Therefore, during reading, the first vertical group of row Z of the first group is read first, then the second vertical group of row Z of the first group is read, and so on. This process continues until the coefficient matrix of the response function is reached. All coefficients; and for each vertical group read from a set of Z rows, immediately adjust it based on the centroid offset matrix. and response function coefficient matrix product matrix The centroid offset matrix obtained in step 4 Perform corresponding operations on the elements at the corresponding positions in the matrix to finally obtain the product matrix. ; Step 6: Based on the initial matrix of wave aberration control voltage set in Step 1.1). The voltage gain matrix set in step 1.3) and the product matrix calculated in step 5 Using equation [1], the wave aberration control voltage matrix for this operation is calculated. Equation [1] is:

[0011] In formula [1]: This represents the wave aberration control voltage matrix for this operation. Matrix; This represents the wave aberration control voltage matrix obtained in the previous calculation adjacent to the current one. When this is the first calculation, Take the initial matrix of wave aberration control voltage set in step 1.1). ; Represents the product matrix With voltage gain matrix The Hadamard; Step 7: Calculate the wave aberration control voltage matrix obtained in Step 6. Adjust the surface shape of the deformable mirror in the optical system, and then acquire the wavefront image after adjusting the surface shape of the deformable mirror; Step 8: The FPGA execution program determines whether it has received the set wavelet aberration correction end command; if not, it returns to step 2; if yes, the wavelet aberration correction ends and the image calculation processing is completed.

[0012] Furthermore, the wavefront aberration correction system includes a wavefront sensor, a wavefront aberration correction control system, N correction voltage loops, and an optical system connected in sequence to form a loop; the wavefront sensor includes multiple Hartmann sub-apertures; the wavefront sensor is used to acquire the wavefront image generated by the optical system and transmit it to the wavefront aberration correction control system; the wavefront aberration correction control system is used to calculate and adjust the wavefront aberration control voltage in each of the correction voltage loops according to the transmitted wavefront image, so as to change the surface profile of the deformable mirror in the optical system to achieve wavefront aberration correction and generate a wavefront image after adjusting the surface profile of the deformable mirror.

[0013] Further, in step 1.2), the coefficients with the same row number in each H column of each parallel coefficient matrix are concatenated in parallel and stored one by one into the corresponding storage cells in a row of DDR3. When storing in DDR3 in the form of n groups, each group with Z rows and each row storing M coefficients, each parallel coefficient matrix is ​​first stored in parallel into NOR flash, and then concatenated in parallel from NOR flash and stored into DDR3. H is equal to the ratio of the word length of a DDR3 memory cell to the word length of a NOR flash memory cell.

[0014] Further, in step 1.2), the step of first storing each parallel coefficient matrix in parallel into NOR flash, and then concatenating them in parallel from NOR flash and storing them into DDR3, specifically involves: First, store the coefficients in one column of the parallel coefficient matrix into a corresponding row in the NOR flash, and then store the n coefficients... The coefficients in each column of the parallel coefficient matrix are stored sequentially into N rows of NOR flash, and stored in NOR flash as n groups of C rows each with M coefficients per row. Then, in parallel, each group of C rows and M coefficients stored in NOR flash is processed, and the coefficients are concatenated into a single row in DDR3, with each group consisting of H rows of coefficients. The coefficients stored in the memory cells with the same column order in each H row are concatenated sequentially into the high-to-low bits of the corresponding memory cell in the DDR3 row. Since each group of C rows contains M coefficients, when concatenating them into H groups, if the last group has fewer than H rows, the memory cells corresponding to the missing rows are marked as invalid data and padded to complete the H rows before concatenation. Finally, the data is stored in DDR3 as n groups, each group containing Z rows and M coefficients per row. Marking the memory cells corresponding to the missing rows as invalid data and padding them to complete the H rows before concatenation prevents data corruption during storage and retrieval, ensuring the accuracy of stored and retrieved data.

[0015] Furthermore, the NOR flash memory cell has a word length of 32 bits, the DDR3 memory cell has a word length of 128 bits, and in step 1.2), H equals 4.

[0016] Furthermore, in order to improve the parallel processing speed without wasting too many hardware resources, in step 3, the set number of packets b should meet the following condition: the value of a / b can be divided evenly by the data length D that can be stored in each row of DDR3.

[0017] The beneficial effects of this invention are: (1) The present invention provides a fast image calculation and processing method based on wavelet aberration correction, which stores the response function coefficient matrix in DDR3. First, divide the matrix into n groups, grouping each of the C columns into n subgroups. The parallel coefficient matrices are then stored in parallel in NOR flash memory, where they are stored as n groups, each with C rows and M coefficients per row. Then, the coefficients in each group of C rows and M coefficients stored in NOR flash memory are concatenated and stored in a single row of DDR3 memory, with each H rows of coefficients being stored sequentially from the high to the low bits of the corresponding storage cell in a single row of DDR3 memory. This results in n groups, each with Z rows and M coefficients per row. Due to this grouping, the storage speed is significantly improved. Furthermore, in this invention, the calculation of the centroid offset matrix... First, the read wavefront images were windowed and divided into packets. For the resulting b windowed image packets, the centroid offsets of each windowed image in the X and Y directions were calculated in parallel, which also improved the calculation of the centroid offset matrix. The computational speed; Meanwhile, because the response function coefficient matrix is ​​used in this invention The coefficients are stored in DDR3 as n groups, each with Z rows and M coefficients per row. In step 5, the FPGA reads the response function coefficient matrix from DDR3. In DDR3, each vertical group is defined to contain a / b coefficients. The process involves reading the first vertical group of the first Z rows, then the second vertical group of the first Z rows, and so on, until the response function coefficient matrix is ​​obtained. All coefficients are read in the same way as in the product matrix. This allows for the calculation of the product matrix. At that time, for each vertical group in a Z-row, it can be immediately aligned with the centroid offset matrix obtained in step 4. The corresponding elements at the specified positions are processed accordingly, thus improving the speed of matrix operations during the calculation of the wave aberration control voltage matrix; furthermore, in step 5, the FPGA reads the response function coefficient matrix from DDR3. At this time, Z rows of data can be read in one address, reducing the significant time spent on addressing during each address jump in DDR3, thus reducing DDR3 addressing time and improving the speed of reading data from DDR3; therefore, when using the image fast calculation and processing method based on wavefront correction of this invention to perform image calculation and processing on wavefront images, the response function coefficient matrix The matrix operation speed in the process of storing, reading and calculating the wave aberration control voltage matrix has been improved, and its real-time performance is better, which can meet the real-time requirements of wave aberration correction. In addition, the image fast calculation and processing method based on wavelet aberration correction of the present invention has low power consumption and requires less capacity for storage, reading and calculation, which reduces the computing power requirements of the hardware. In the present invention, the fast image calculation and processing based on wavelet aberration correction is completed through the cooperation of three hardware components: NOR flash, DDR3 and FPGA. Therefore, it can meet the requirements of high reliability, low power consumption and low latency for storage, reading and calculation when used on satellite platforms. In summary, this invention solves the technical problems of existing image storage and retrieval schemes, which suffer from slow storage and retrieval speeds, poor real-time performance, and inability to simultaneously meet the real-time requirements of wavelet aberration correction as well as the requirements of high reliability, low power consumption, and low latency for storage, retrieval, and computation when used on satellite platforms.

[0018] (2) In this invention, when setting the number of packets in step 3, the value of a / b is such that it can be divided by the data length D that can be stored in each row of DDR3. This improves the parallel processing speed without wasting too much hardware resources.

[0019] (3) By adopting the fast image calculation and processing method based on wave aberration correction of the present invention, the time for calculating the wave aberration control voltage matrix in a single operation is shortened from the original 160ms to 10.3ms, and the frame rate is also increased to 100 frames per second, which greatly improves the efficiency of image calculation and processing.

[0020] (4) The image fast calculation and processing method based on wavelet aberration correction of the present invention has fast storage, retrieval and calculation speed, good real-time performance, low power consumption and high reliability, providing technical support for on-board wavelet aberration correction, and also providing a reference for on-board image fast calculation and processing in other fields. Attached Figure Description

[0021] Figure 1 This is a flowchart of an embodiment of the fast image calculation and processing method based on wavelet aberration correction of the present invention; Figure 2 In an embodiment of the image fast calculation and processing method based on wavelet aberration correction of the present invention, in step 1.2), the response function coefficient matrix is... The coefficients are split and stored in parallel in NOR flash. Then, the coefficients of each group (C rows, M coefficients per row) stored in NOR flash are concatenated in parallel and stored in DDR3. In step 5, the FPGA reads the response function coefficient matrix stored in step 1.2) from DDR3. At that time, a schematic diagram of its storage and retrieval strategy is shown; Figure 3 In this embodiment of the fast image calculation and processing method based on wavefront aberration correction of the present invention, the wavefront image is windowed and divided into packets, and the response function coefficient matrix is ​​processed. The diagram illustrates the effect of splitting the image and using the fast image calculation and processing method based on wavelet aberration correction of this invention to shorten the time for calculating the wavelet aberration control voltage matrix in a single operation. Detailed Implementation

[0022] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0023] See Figure 1This invention discloses a rapid image processing method based on wavefront aberration correction. The wavefront aberration correction system comprises a wavefront sensor, a wavefront aberration correction control system, N correction voltage loops, and an optical system, connected sequentially in a loop. The wavefront sensor includes multiple Hartmann sub-apertures. The wavefront sensor acquires a wavefront image generated by the optical system and transmits it to the wavefront aberration correction control system. The wavefront aberration correction control system calculates and adjusts the wavefront aberration control voltage in each correction voltage loop based on the transmitted wavefront image, thereby changing the surface profile of the deformable mirror in the optical system to correct the wavefront aberration and generate a wavefront image after adjusting the deformable mirror surface profile. This rapid image processing method based on wavefront aberration correction includes the following steps: Step 1: Initialize the wavelet aberration correction control system parameters, specifically including: 1.1) Based on the number of correction voltage loops N and the calibration results of the wave aberration correction system, set the initial matrix of the wave aberration control voltage in the NOR flash. The above wave aberration control voltage initial matrix for The matrix; in this embodiment, the number of correction voltage loops N is 954, and the initial matrix of the wave aberration control voltage is set. for Matrix; 1.2) Based on Hartmann's number of sub-apertures 'a' and the number of correction voltage loops 'N', set the response function coefficient matrix of centroid offset and wave aberration control voltage adjustment. The above response function coefficient matrix The size is , Then, divide it into n groups, grouping each of columns C into a single group. The parallel coefficient matrix, the above See also Figure 3 In this embodiment, the number of sub-apertures 'a' in Hartmann is 749. The set response function coefficient matrix The size is If C is 18, then n is 53. response function coefficient matrix Divided into 53 The parallel coefficient matrices are then constructed; next, the coefficients with the same row number in each H column of each parallel coefficient matrix are concatenated in parallel and stored one-to-one in the corresponding storage cells of a row in DDR3. In DDR3, this is stored as n groups, each group with Z rows, and each row storing M coefficients; 2≤H≤C; Z is the floor value obtained by rounding up the quotient of C divided by H; see [link to relevant documentation]. Figure 2In this embodiment, the parallel coefficient matrices are first stored in NOR flash in parallel, and then concatenated in parallel from NOR flash and stored in DDR3. Specifically: First, store the coefficients in one column of the parallel coefficient matrix into a corresponding row in the NOR flash, and then store the n coefficients... The coefficients in each column of the parallel coefficient matrix are stored sequentially in N rows of NOR flash, and stored in NOR flash as n groups of C rows each with M coefficients per row; in this embodiment, they are stored in NOR flash as 53 groups of 18 rows each with 1498 coefficients per row. Then, in parallel, each group of C rows and M coefficients stored in NOR flash is processed, and the coefficients are concatenated into a single row in DDR3 according to the method of concatenating H rows of coefficients. The coefficients stored in the memory cells with the same column order in each H row are concatenated sequentially according to the row number and stored in the high-to-low bits of the corresponding memory cell in the DDR3 row. When concatenating each group of C rows and M coefficients, if the last group has fewer than H rows, the memory cells corresponding to the missing rows are marked as invalid data and H rows are filled in before concatenation. Finally, the data is stored in DDR3 in the form of n groups, Z rows, and M coefficients per row. Marking the memory cells corresponding to the missing rows as invalid data and filling H rows before concatenation prevents data corruption during storage and retrieval, ensuring the accuracy of stored and retrieved data. Here, H is equal to the ratio of the word length of the DDR3 memory cell to the word length of the NOR flash memory cell; see [link to relevant documentation]. Figure 2 In this embodiment, the word length of the NOR flash memory cell is 32 bits and the word length of the DDR3 memory cell is 128 bits. Therefore, H equals 4 and Z equals 5. When splicing, splicing is performed in groups of 4. Each group consists of 18 rows and each row contains 1498 coefficients. The splicing is stored in the 5 rows of the DDR3 memory, with each row containing 1498 coefficients. In this embodiment, the first storage cell of the first row of DDR3 stores the following content: the high 32 bits store the first coefficient of the first row of the first 1498 coefficients in the first 18 rows of NOR flash; the second high 32 bits store the first coefficient of the second row of the first 1498 coefficients in the first 18 rows of NOR flash; the second low 32 bits store the first coefficient of the third row of the first 1498 coefficients in the first 18 rows of NOR flash; the low 32 bits store the first coefficient of the fourth row of the first 1498 coefficients in the first 18 rows of NOR flash; and so on. In this embodiment, the second low 32 bits and the low 32 bits in each storage cell of the fifth row of DDR3 are marked as invalid data because they lack coefficients. The same applies to the second low 32 bits and the low 32 bits in the 10th row, ..., the 265th (53×5)th row. 1.3) Based on the actual hardware environment of the wavelet aberration correction system, set the voltage gain matrix in the NOR flash. The voltage gain matrix mentioned above The size is In this embodiment, the voltage gain matrix The size is ; The order of steps 1.1), 1.2), and 1.3) above can be interchanged and is not important. Step 2: Read the wavefront image; Step 3: See Figure 3 Based on the number of sub-apertures *a* in Hartmann's algorithm, the read wavefront image is windowed and packetized to obtain *b* windowed image packets. Each windowed image packet contains *a / b* windowed images, where *b* is the set number of packets, and *a / b* is an integer. To improve parallel processing speed without wasting excessive hardware resources, the set number of packets *b* preferably meets the following condition: the value of *a / b* is divisible by the data length *D* that can be stored in each row of DDR3. In this embodiment, *b* is set to 7, and *a / b* is 107. Preferably, D is an integer multiple of 107; Step 4: For each windowed image packet obtained in Step 3, calculate the centroid offset of each windowed image in the X and Y directions in parallel to obtain the centroid offset matrix. The above centroid offset matrix The size is In this embodiment, the centroid offset matrix The size is ; Step 5: The FPGA reads the response function coefficient matrix stored in step 1.2) from DDR3. In DDR3, each vertical column contains a / b coefficients. Therefore, during reading, the first column of row Z of the first group is read first, then the second column of row Z of the first group is read, and so on. This process continues until the coefficient matrix of the response function is reached. All coefficients; see all coefficients in; see Figure 2 In this embodiment, each vertical group in DDR3 is defined to include 107 coefficients, then the response function coefficient matrix... The DDR3 memory is stored in 14 vertical groups. The reading method starts from the first row and reads the first vertical group of rows 1 to 5 in sequence. After that, it starts from the first row again and reads the second vertical group of rows 1 to 5 in sequence, and so on. After reading the first group of 5 rows, it reads the second group of 5 rows in sequence, until the response function coefficient matrix mentioned above is read. All coefficients in the matrix are calculated. And for each vertical group read from a set of Z rows, the centroid offset matrix is ​​immediately calculated. and response function coefficient matrix product matrix The centroid offset matrix obtained in step 4 The corresponding elements at the specified positions are then subjected to corresponding operations. In this embodiment, the operation is performed immediately after each vertical group of 5 rows is read, ultimately yielding the product matrix. Product matrix The size is ; Step 6: Based on the initial matrix of wave aberration control voltage set in Step 1.1). The voltage gain matrix set in step 1.3) and the product matrix calculated in step 5 Using equation [1], the wave aberration control voltage matrix for this operation is calculated. Equation [1] is:

[0024] In formula [1]: This represents the wave aberration control voltage matrix for this operation. The matrix, in this embodiment is Matrix; This represents the wave aberration control voltage matrix obtained in the previous calculation adjacent to the current one. When this is the first calculation, Take the initial matrix of wave aberration control voltage set in step 1.1). ; Represents the product matrix With voltage gain matrix The Hadamard; Step 7: Calculate the wave aberration control voltage matrix obtained in Step 6. Adjust the surface shape of the deformable mirror in the optical system, and then acquire the wavefront image after adjusting the surface shape of the deformable mirror; Step 8: The FPGA execution program determines whether it has received the set wavelet aberration correction end command; if not, it returns to step 2; if yes, the wavelet aberration correction ends and the image calculation processing is completed.

[0025] In summary, this invention provides a fast image calculation and processing method based on wavelet aberration correction, which utilizes the response function coefficient matrix... The splitting and splicing storage of the response function coefficient matrix Storing it in DDR3 in a specific way can not only improve the response function coefficient matrix The storage speed in DDR3, and also laid the foundation for improving the speed of matrix operations in the subsequent calculation of the waveform aberration control voltage matrix; response function coefficient matrix The response function coefficient matrix is ​​stored in DDR3 in a specific manner, and then read from DDR3 in a specific manner by the subsequent FPGA. This improves the speed of matrix operations in calculating the wave aberration control voltage matrix; additionally, it increases the speed of calculation of the centroid offset matrix. First, the read wavefront image was windowed and divided into packets, which improved the calculation of the centroid offset matrix. The computation speed is improved; subsequent experiments show that the image fast calculation processing method based on wavelet aberration correction of this invention reduces the time for a single calculation of the wavelet aberration control voltage matrix from 160ms to 10.3ms. See [link to relevant documentation]. Figure 3 The frame rate has also been increased to 100 frames per second, significantly improving the efficiency of image computation and processing. This invention, based on a wavefront correction-based rapid image computation and processing method, utilizes the collaborative hardware of NOR flash, DDR3, and FPGA to complete rapid image computation and processing based on wavefront correction. It features fast storage, retrieval, and computation speeds, good real-time performance, low power consumption, and high reliability, providing technical support for onboard wavefront correction and serving as a reference for rapid onboard image computation and processing in other fields. This invention is applicable to image processing using Hartmann wavefront correction.

Claims

1. A method for rapid image calculation and processing based on wave aberration correction, the method is implemented based on a wave aberration correction system, characterized in that: The following steps are involved: Step 1: Initialize the parameters of the wave aberration correction control system, including: 1.1) According to the number of correction voltage loops N and the calibration results of the wave aberration correction system, set the initial matrix of wave aberration control voltage in NOR flash , the wave aberration control voltage initial matrix for Matrix of 1.2) According to the number of Hartmann subapertures a and the number of correction voltage loops N, the response function coefficient matrix of the center of mass offset and the wave aberration control voltage adjustment is set: , the response function coefficient matrix The size is , ; Then split each C column into n groups The parallel coefficient matrix, ; Then, in parallel, the coefficients with the same row number in each H column of each parallel coefficient matrix are spliced ​​and stored one by one in the corresponding storage unit in a row of DDR3 in sequence, and stored in DDR3 in the form of n groups, each group has Z rows, and each row stores M coefficients; 2≤H≤C; Z is the value obtained by rounding up the quotient of C divided by H; 1.3) According to the actual hardware environment of the wave aberration correction system, set the voltage gain matrix in the NOR flash , the voltage gain matrix The size is ; Step 2: Read the wavefront image; Step 3: According to the number a of Hartmann's sub-apertures, the read wavefront image is windowed and packaged to obtain b windowed image packages, each of which contains a / b windowed images, where b is the set number of packages and the value of a / b is an integer; Step 4: For each of the windowed image packages obtained in step 3, the centroid offsets of each windowed image in each package in the X and Y directions are calculated in parallel to obtain a centroid offset matrix , the centroid offset matrix The size is ; Step 5: FPGA reads the response function coefficient matrix stored in step 1.2) from DDR3 ; Define that each vertical group in DDR3 includes a / b coefficients. When reading, first read the first vertical group of the first group of Z rows, then read the second vertical group of the first group of Z rows, and so on. After reading the first group of Z rows, read the second group of Z rows, and so on, until the response function coefficient matrix is ​​read. All coefficients in ; and each time a vertical group in a group of Z rows is read, it is immediately based on the centroid offset matrix and the response function coefficient matrix The product matrix , and the centroid offset matrix obtained in step 4 The corresponding operations are performed on the elements at the corresponding positions in the matrix, and the product matrix is ​​finally calculated. ; Step 6: Set the initial matrix of wave aberration control voltages according to step 1.1) , the voltage gain matrix set in step 1.3) And the product matrix calculated in step 5 , using formula [1], calculate the wave aberration control voltage matrix , formula [1] is: ； In formula [1]: The wave aberration control voltage matrix is ​​represented as Matrix of It represents the wave aberration control voltage matrix obtained by the previous calculation adjacent to this one. When this is the first calculation, Take the initial matrix of wave aberration control voltage set in step 1.1) ; Represents the product matrix With the voltage gain matrix The Hadamard; Step 7: Based on the wave aberration control voltage matrix calculated in step 6 , adjusting the surface shape of the deformable mirror in the optical system, and then obtaining a wavefront image after the deformable mirror surface shape is adjusted; Step 8: The FPGA executes the program to determine whether the set wave aberration correction end instruction is received; if not, it returns to step 2; if so, the wave aberration correction is ended and the image calculation processing is completed.

2. The image fast calculation and processing method based on wave aberration correction according to claim 1, characterized in that: The wave aberration correction system includes a wavefront sensor, a wave aberration correction control system, N correction voltage circuits and an optical system which are connected in sequence to form a circuit; the wavefront sensor includes a plurality of Hartmann sub-apertures; the wavefront sensor is used to obtain the wavefront image generated by the optical system and transmit it to the wave aberration correction control system; the wave aberration correction control system is used to calculate and adjust the wave aberration control voltage in each correction voltage circuit according to the transmitted wavefront image, so that the surface shape of the deformable mirror in the optical system is changed to realize the correction of the wave aberration and generate a wavefront image after the deformable mirror surface shape is adjusted.

3. The image rapid calculation and processing method based on wave aberration correction according to claim 1, characterized in that: In step 1.2), the coefficients with the same row number in each H column of each parallel coefficient matrix are spliced ​​in parallel and stored one by one in the corresponding storage unit in a row of DDR3. When the coefficients are stored in DDR3 in the form of n groups with Z rows in each group and M coefficients in each row, the parallel coefficient matrices are first stored in parallel in NOR flash, and then spliced ​​in parallel from NOR flash and stored in DDR3; The H is equal to the ratio of the word length of the DDR3 storage unit to the word length of the NOR flash storage unit.

4. The image fast calculation and processing method based on wave aberration correction according to claim 3 is characterized in that: In step 1.2), the parallel coefficient matrices are first stored in parallel in the NOR flash, and then spliced ​​and stored in parallel in the DDR3 from the NOR flash, specifically: First, store the coefficients of a column in the parallel coefficient matrix into a row in the NOR flash. The coefficients of each column in the parallel coefficient matrix are stored one by one in N rows in the NOR flash in sequence, and are stored in the NOR flash in the form of n groups, each group has C rows, and each row has M coefficients; Then, in parallel, each group of C rows of M coefficients stored in the NOR flash are spliced ​​into one row of DDR3 in the manner of splicing every H rows of coefficients. The coefficients stored in the storage cells with the same column order in each H rows are spliced ​​in sequence from high to low bits in the corresponding storage cells in a row of DDR3 according to the row number. Moreover, when each group of C rows of M coefficients is spliced ​​as a group with every H rows of coefficients, if the last group has less than H rows, the corresponding storage cells in the missing rows are marked as invalid data to fill up H rows, and then spliced. Finally, it is stored in DDR3 in the form of n groups of Z rows with M coefficients stored in each row.

5. The image fast calculation and processing method based on wave aberration correction according to claim 4 is characterized in that: The NOR flash storage unit word length is 32 bits, the DDR3 storage unit word length is 128 bits, and in step 1.2), H is equal to 4.

6. The image fast calculation and processing method based on wave aberration correction according to claim 1, characterized in that: In step 3, the setting of the number of subpackets b should meet the following conditions: the value of a / b can evenly divide the data length D that can be stored in each row in DDR3.

Images