A fast image processing method based on wavefront aberration correction

Abstract

The present invention relates to an image processing method, and in particular to an image fast calculation and processing method based on wave aberration correction, which solves the problem that the existing solution is slow and cannot simultaneously meet the low power consumption and low delay requirements of wave aberration correction and satellite platform use. The method comprises the following steps: Step 1: Parameter initialization: 1.1) Setting the initial matrix of wave aberration control voltage; 1.2) Setting the response function coefficient matrix, splitting and storing it in DDR3; 1.3) Setting the voltage gain matrix; Step 2: Reading the wavefront image; Step 3: Windowing and packetization; Step 4: Calculating the centroid offset matrix; Step 5: Calculating the product matrix; Step 6: Calculating the wave aberration control voltage matrix of this time; Step 7: Obtaining the wavefront image after adjusting the deformable mirror type; Step 8: Determining whether an end instruction is received; If not, returning to step 2; If so, completing. The present invention is suitable for image processing of Hartmann wavefront correction.

Description

Technical Field

[0001] The invention relates to an image processing method, in particular to an image fast calculation processing method based on wave aberration correction. Background Art

[0002] Image storage and reading refers to converting the received data to be processed into a data stream such as a matrix according to the rules, storing it in the medium, and then reading the data according to the rules for image processing. Commonly used storage media include NOR flash, DDR, etc. If the storage and reading of images is regarded as a pipeline, data format conversion is required between different storage media to complete data transmission.

[0003] As the requirements for space target detection are increasing, the real-time requirements for high-speed storage and reading technology of images with space targets as the background are also getting higher and higher. The rapid development of integrated circuit technology provides hardware support for solving this problem; and how to cleverly use advanced integrated circuit hardware resources to achieve high-speed storage and reading of images with high real-time performance has become a challenging task. In recent years, high-speed real-time sensor image processing systems based on FPGA and DDR technology have received widespread attention. These systems can achieve high-speed image acquisition, preprocessing and storage, providing strong support for various applications.

[0004] At present, the common storage schemes are as follows: (1) High-speed remote image data storage system based on LVDS. This system uses FPGA chip as the control core, and adopts low voltage differential signal technology (LVDS) interface deserialization and driver chip to ensure the effective reception of remote data; it uses alternating dual-plane FLASH programming to control the storage of image data. (2) In response to the demand for high-speed image data FPGA large-capacity storage, some scholars have proposed an invalid block management strategy based on high-speed parallel FLASH large-capacity data storage structure. (3) Other scholars have proposed a customized image encoding method for approximate storage substrate. They found that indiscriminately storing encoded images in approximate memory will produce unacceptable and uncontrollable quality problems. (4) Some researchers have explored the problem of using DDR for image storage. (5) There is also a paper that proposes a design method for high-speed image data transmission system based on DDR2. The system designed by this design method uses FPGA internal FIFO resources and off-chip DDR2 hierarchical cache mechanism to meet the high throughput requirements of images and large capacity requirements of data cache. All of the above storage solutions focus on the high-speed storage of image data and provide specific solutions. The common point of these solutions is that they all emphasize the importance of high speed, large capacity and reliability, and all use FPGA as the control core of the storage system, and all use FPGA parallel processing capabilities and flexibility to achieve high-speed and large-capacity data storage. The difference is that they target different application needs and problems, and use different specific technologies and methods. The shortcomings of the above solutions are: first, although the storage and reading speed has been improved, it is still slow, and the real-time performance is still poor, which cannot meet the real-time requirements of wave aberration correction; second, most of them are used for ground equipment, requiring large-capacity GPU servers, while the hardware resources on the satellite platform are limited, and the capacity that can be allocated to image storage, reading and calculation is limited. Therefore, it cannot meet the low power consumption and low latency required for application on the satellite platform, and it is difficult to be directly applied on the satellite platform; although some can be directly applied on the satellite platform, they cannot meet the requirements of wave aberration correction due to their slow speed and poor real-time performance.

[0005] In addition, some scholars have proposed a storage solution based on FPGA to control Nand Flash array to achieve high-speed pipeline storage. This solution writes image data into the FLASH storage array through the CameraLink input communication interface, and uses DMA transmission technology to upload the stored image file to the computer hard disk for further processing; at the same time, the SDRAM video memory is used to refresh the data in real time, and the FPGA constructs the corresponding VGA signal, finally realizing the real-time display of 100MB / s images. However, this solution only realizes the real-time display of images, and does not calculate or interpret the image data.

[0006] Some people also proposed to study image storage and reading based on DDR4-based photoelectric image real-time storage technology. They first introduced the key technologies and working principles of DDR4; then, in the interface module, they studied the three-cache interface technology to realize the function of real-time reading and writing data of multiple devices; and in the read and write operation modules, they proposed a cycle dispersion refresh algorithm to solve the instruction lag problem. However, this solution only improves the response speed at the beginning, and does not speed up the subsequent storage, reading and calculation speed.

[0007] In summary, it is an urgent problem to develop a method for fast image computing and processing that can quickly store, read and calculate images, so that it can simultaneously meet the real-time requirements of wave aberration correction and the high reliability, low power consumption and low latency requirements of storage, reading and calculation when used on satellite platforms.

[0008] The following is an explanation of the English vocabulary involved:

[0009] NOR flash: A non-volatile memory with fast read capability, commonly used to store firmware and operating systems;

[0010] DDR: Double Data Rate SDRAM, double data rate synchronous dynamic random access memory;

[0011] DDR2: Double Data Rate 2 SDRAM, double data rate second generation synchronous dynamic random access memory;

[0012] DDR3: Double Data Rate 3 SDRAM, double data rate three generation synchronous dynamic random access memory;

[0013] DDR4: Double Data Rate 4 SDRAM, double data rate four-generation synchronous dynamic random access memory;

[0014] FPGA: Field Programmable Gate Array, field programmable logic gate array;

[0015] LVDS: Low Voltage Differential Signaling, low voltage differential signal;

[0016] FLASH: A non-volatile memory that can read and write data quickly;

[0017] FIFO: First In First Out, a data buffer whose main feature is that data is written in sequence and then output in the same order, that is, the data entered first is taken out first;

[0018] GPU: Graphics Processing Unit, graphics processing unit;

[0019] Nand Flash: NAND flash memory is a non-volatile memory technology mainly used for data storage;

[0020] CameraLink: A serial communication protocol specifically targeted at machine vision applications;

[0021] DMA: Direct Memory Access is a hardware feature that allows certain hardware subsystems to directly access the system's memory without going through the central processing unit (CPU);

[0022] SDRAM: Synchronous Dynamic Random-Access Memory, synchronous dynamic random access memory;

[0023] VGA: Video Graphics Array, a standard for video transmission using analog signals. Summary of the invention

[0024] The purpose of the present invention is to solve the technical problems that the existing image storage and reading schemes have slow storage and reading speeds, poor real-time performance, and cannot simultaneously meet the real-time requirements of wave aberration correction and the high reliability, low power consumption, and low latency requirements of storage, reading, and calculation when used on satellite platforms, and to provide an image fast calculation and processing method based on wave aberration correction.

[0025] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0026] A method for rapid image calculation and processing based on wave aberration correction is implemented based on a wave aberration correction system, and its special feature is that it includes the following steps:

[0027] Step 1: Initialize the parameters of the wave aberration correction control system, including:

[0028] 1.1) According to the number of correction voltage loops N and the calibration results of the wave aberration correction system, set the initial matrix of wave aberration control voltage in NOR flash , the wave aberration control voltage initial matrix for Matrix of

[0029] 1.2) According to the number of Hartmann subapertures a and the number of correction voltage loops N, the response function coefficient matrix of the center of mass offset and the wave aberration control voltage adjustment is set: , the response function coefficient matrix The size is , ; Then split each C column into n groups The parallel coefficient matrix, ; Then, in parallel, the coefficients with the same row number in each H column of each parallel coefficient matrix are spliced ​​and stored one by one in the corresponding storage unit in a row of DDR3 in sequence, and stored in DDR3 in the form of n groups, each group has Z rows, and each row stores M coefficients; 2≤H≤C; Z is the value obtained by rounding up the quotient of C divided by H;

[0030] 1.3) According to the actual hardware environment of the wave aberration correction system, set the voltage gain matrix in the NOR flash , the voltage gain matrix The size is ;

[0031] Step 2: Read the wavefront image;

[0032] Step 3: According to the number a of Hartmann's sub-apertures, the read wavefront image is windowed and packaged to obtain b windowed image packages, each of which contains a / b windowed images, where b is the set number of packages and the value of a / b is an integer;

[0033] Step 4: For each of the windowed image packages obtained in step 3, the centroid offsets of each windowed image in each package in the X and Y directions are calculated in parallel to obtain a centroid offset matrix , the centroid offset matrix The size is ;

[0034] Step 5: FPGA reads the response function coefficient matrix stored in step 1.2) from DDR3 ; Define that each vertical group in DDR3 includes a / b coefficients. When reading, first read the first vertical group of the first group of Z rows, then read the second vertical group of the first group of Z rows, and so on. After reading the first group of Z rows, read the second group of Z rows, and so on, until the response function coefficient matrix is ​​read. All coefficients in ; and each time a vertical group in a group of Z rows is read, it is immediately based on the centroid offset matrix and the response function coefficient matrix The product matrix , and the centroid offset matrix obtained in step 4 The corresponding operations are performed on the elements at the corresponding positions in the matrix, and the product matrix is ​​finally calculated. ;

[0035] Step 6: Set the initial matrix of wave aberration control voltages according to step 1.1) , the voltage gain matrix set in step 1.3) And the product matrix calculated in step 5 , using formula [1], calculate the wave aberration control voltage matrix , formula [1] is:

[0036]

[0037] In formula [1]: The wave aberration control voltage matrix is ​​represented as Matrix of It represents the wave aberration control voltage matrix obtained by the previous calculation adjacent to this one. When this is the first calculation, Take the initial matrix of wave aberration control voltage set in step 1.1) ; Represents the product matrix With the voltage gain matrix The Hadamard;

[0038] Step 7: Based on the wave aberration control voltage matrix calculated in step 6 , adjusting the surface shape of the deformable mirror in the optical system, and then obtaining a wavefront image after the deformable mirror surface shape is adjusted;

[0039] Step 8: The FPGA executes the program to determine whether the set wave aberration correction end instruction is received; if not, it returns to step 2; if so, the wave aberration correction is ended and the image calculation processing is completed.

[0040] Furthermore, the wave aberration correction system includes a wavefront sensor, a wave aberration correction control system, N correction voltage circuits and an optical system which are connected in sequence to form a circuit; the wavefront sensor includes a plurality of Hartmann sub-apertures; the wavefront sensor is used to obtain a wavefront image generated by the optical system and transmit it to the wave aberration correction control system; the wave aberration correction control system is used to calculate and adjust the wave aberration control voltage in each correction voltage circuit according to the transmitted wavefront image, so that the surface shape of the deformable mirror in the optical system changes, so as to realize the correction of the wave aberration and generate a wavefront image after the deformable mirror surface shape is adjusted.

[0041] Furthermore, in step 1.2), the coefficients with the same row number in each H column of each parallel coefficient matrix are spliced ​​in parallel and stored one by one in the corresponding storage units in a row of DDR3. When the coefficients are stored in DDR3 in the form of n groups with each group having Z rows and each row storing M coefficients, the parallel coefficient matrices are first stored in parallel in NOR flash, and then spliced ​​in parallel from NOR flash and stored in DDR3;

[0042] The H is equal to the ratio of the word length of the DDR3 storage unit to the word length of the NOR flash storage unit.

[0043] Furthermore, in step 1.2), the parallel coefficient matrices are first stored in parallel in the NOR flash, and then spliced ​​and stored in parallel in the DDR3 from the NOR flash, specifically:

[0044] First, store the coefficients of a column in the parallel coefficient matrix into a row in the NOR flash. The coefficients of each column in the parallel coefficient matrix are stored one by one in N rows in the NOR flash in sequence, and are stored in the NOR flash in the form of n groups, each group has C rows, and each row has M coefficients;

[0045] Then, in parallel, each group of C rows and M coefficients stored in each row of NOR flash are spliced ​​and stored in a row of DDR3 in the manner of splicing coefficients in every H rows. The coefficients stored in the storage cells with the same column order in every H rows are spliced ​​and stored in the high to low bits of the corresponding storage cells in a row of DDR3 in sequence according to the row number. Moreover, when each group of C rows and M coefficients are spliced ​​as a group of coefficients in every H rows, if the last group is less than H rows, the corresponding storage cells in the missing rows are marked as invalid data to fill H rows and splice, and finally stored in DDR3 in the form of n groups and Z rows and M coefficients in each row. Marking the corresponding storage cells in the missing rows as invalid data to fill H rows and splicing can prevent data confusion during storage and reading, and ensure the accuracy of stored and read data.

[0046] Furthermore, the word length of the NOR flash storage unit is 32 bits, and the word length of the DDR3 storage unit is 128 bits. In step 1.2), H is equal to 4.

[0047] Furthermore, in order to improve the parallel processing speed without wasting too much hardware resources, in step 3, the setting of the number of subpackets b should meet the following conditions: the value of a / b can divide the data length D that can be stored in each row in DDR3.

[0048] The beneficial effects of the present invention are:

[0049] (1) The present invention is based on the image fast calculation and processing method of wave aberration correction, storing the response function coefficient matrix in DDR3 When , first split the matrix into n groups with each C column as a group. The parallel coefficient matrices are then stored in parallel in the NOR flash, and stored in the NOR flash in the form of n groups, each with C rows and M coefficients per row; and then the coefficients of each group, each with C rows and M coefficients per row stored in the NOR flash are stored in parallel in a row of DDR3 in the manner of splicing each H rows of coefficients, and the coefficients stored in the storage units with the same column order in each H rows are sequentially spliced ​​and stored in the high to low positions of the corresponding storage units in a row of DDR3 according to the row number, and stored in the DDR3 in the form of n groups, each with Z rows and M coefficients per row. Due to the grouping, the storage speed is greatly improved; and in the present invention, when calculating the centroid offset matrix Before the wavefront image is read, it is windowed and packaged. For the b windowed image packages obtained, the centroid offset of each windowed image in the X and Y directions in each package is calculated in parallel, which also improves the calculation of the centroid offset matrix. The calculation speed of

[0050] At the same time, since the response function coefficient matrix The data is stored in DDR3 as n groups with Z rows per group and M coefficients per row. In step 5, the FPGA reads the response function coefficient matrix from DDR3. When defining each vertical group in DDR3 to include a / b coefficients, the first vertical group of the first group of Z rows is read first, then the second vertical group of the first group of Z rows is read, and so on, after reading the first group of Z rows, and then the second group of Z rows is read, until the response function coefficient matrix is ​​read In this way, when calculating the product matrix When reading a vertical group in a group of Z rows, you can immediately compare it with the centroid offset matrix obtained in step 4 The corresponding elements in the corresponding positions in the matrix are operated accordingly, thereby improving the speed of matrix operation in the process of calculating the wave aberration control voltage matrix; and in step 5, the FPGA reads the response function coefficient matrix from DDR3 When the image is processed by the wavefront image fast calculation processing method based on wave aberration correction of the present invention, the response function coefficient matrix The matrix operation speed in the process of storing, reading and calculating the wave aberration control voltage matrix is ​​improved, and its real-time performance is better, which can meet the real-time requirements of wave aberration correction;

[0051] In addition, the image fast calculation processing method based on wave aberration correction of the present invention has low power consumption, and the capacity required to be allocated to its storage, reading and calculation is small, which reduces the computing power requirements of the hardware. In the present invention, the image fast calculation processing based on wave aberration correction is completed by the cooperation of three hardwares: NOR flash, DDR3 and FPGA, so that it can meet the requirements of high reliability, low power consumption and low latency for storage, reading and calculation when used on a satellite platform;

[0052] In summary, the present invention solves the technical problem that the existing image storage and reading solutions have slow storage and reading speeds, poor real-time performance, and cannot simultaneously meet the real-time requirements of wave aberration correction and the high reliability, low power consumption, and low latency requirements of storage, reading, and calculation when used on satellite platforms.

[0053] (2) In the present invention, preferably, when setting the number of subpackets in step 3, the value of a / b can divide the data length D that can be stored in each row in DDR3. This will increase the parallel processing speed without wasting too much hardware resources.

[0054] (3) By using the image fast calculation and processing method based on wave aberration correction of the present invention, the time for a single calculation of the wave aberration control voltage matrix is ​​shortened from the original 160ms to 10.3ms, and the frame rate is also increased to 100 frames per second, which greatly improves the efficiency of image calculation and processing.

[0055] (4) The image rapid calculation and processing method based on wave aberration correction of the present invention has fast storage, reading and calculation speeds, good real-time performance, low power consumption and high reliability. It provides technical support for onboard wave aberration correction and also provides a reference for onboard image rapid calculation and processing in other fields. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 is a flow chart of an embodiment of a method for rapid image calculation and processing based on wave aberration correction of the present invention;

[0057] Figure 2 In the embodiment of the image fast calculation processing method based on wave aberration correction of the present invention, in step 1.2), the response function coefficient matrix The coefficients are split and stored in parallel in NOR flash, and then the coefficients of each group of C rows and M rows stored in NOR flash are spliced ​​in parallel and stored in DDR3. In step 5, FPGA reads the response function coefficient matrix stored in step 1.2) from DDR3. Schematic diagram of its storage and reading strategy;

[0058] Figure 3 In the embodiment of the image fast calculation processing method based on wave aberration correction of the present invention, the wavefront image is windowed and packaged, and the response function coefficient matrix is A schematic diagram of the effect of splitting and adopting the image fast calculation and processing method based on wave aberration correction of the present invention to shorten the time of single calculation of wave aberration control voltage matrix. DETAILED DESCRIPTION

[0059] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0060] See also Figure 1 The present invention discloses a method for rapid image calculation and processing based on wave aberration correction. The wave aberration correction system based on the method includes a wavefront sensor, a wave aberration correction control system, N correction voltage circuits and an optical system which are sequentially connected to form a circuit; the wavefront sensor includes a plurality of Hartmann sub-apertures; the wavefront sensor is used to obtain a wavefront image generated by the optical system and transmit it to the wave aberration correction control system; the wave aberration correction control system is used to calculate and adjust the wave aberration control voltage in each correction voltage circuit according to the transmitted wavefront image, so that the surface shape of the deformable mirror in the optical system changes, so as to realize the correction of the wave aberration and generate a wavefront image after the deformable mirror surface shape is adjusted; the method for rapid image calculation and processing based on wave aberration correction includes the following steps:

[0061] Step 1: Initialize the parameters of the wave aberration correction control system, including:

[0062] 1.1) According to the number of correction voltage loops N and the calibration results of the wave aberration correction system, set the initial matrix of wave aberration control voltage in NOR flash , the above wave aberration control voltage initial matrix for In this embodiment, the number of correction voltage loops N is 954, and the initial matrix of the wave aberration control voltage is set for Matrix of

[0063] 1.2) According to the number of Hartmann subapertures a and the number of correction voltage loops N, the response function coefficient matrix of the center of mass offset and the wave aberration control voltage adjustment is set: , the above response function coefficient matrix The size is , ; Then split each C column into n groups The parallel coefficient matrix of ; See Figure 3In this embodiment, the number of Hartmann subapertures a is 749, then , set the response function coefficient matrix The size is , the above C is 18, then n is 53, The response function coefficient matrix Split into 53 The parallel coefficient matrix; then the coefficients with the same row number in each H column of each parallel coefficient matrix are concatenated in parallel, and stored one by one in the corresponding storage unit in a row of DDR3, and stored in DDR3 in the form of n groups, each group with Z rows, and each row storing M coefficients; the above 2≤H≤C; the above Z is the value obtained by rounding up the quotient of C divided by H; see Figure 2 In this embodiment, each parallel coefficient matrix is ​​first stored in parallel in NOR flash, and then parallel spliced ​​from NOR flash and stored in DDR3, specifically:

[0064] First, store the coefficients of a column in the parallel coefficient matrix into a row in the NOR flash. The coefficients of each column in the parallel coefficient matrix are stored one by one in N rows in the NOR flash in turn, and are stored in the NOR flash in the form of n groups, each group has C rows, and each row has M coefficients; in this embodiment, the coefficients are stored in the NOR flash in the form of 53 groups, each group has 18 rows, and each row has 1498 coefficients;

[0065] Then, in parallel, each group of C rows and M coefficients stored in NOR flash are spliced ​​into one row of DDR3 in the manner of splicing coefficients in every H rows, and the coefficients stored in the storage cells with the same column order in every H rows are spliced ​​in sequence according to the row number and stored in the high to low bits of the corresponding storage cells in one row of DDR3. Moreover, when each group of C rows and M coefficients are spliced ​​as a group of H rows of coefficients, if the last group has less than H rows, the corresponding storage cells in the missing rows are marked as invalid data to fill H rows for splicing, and finally stored in DDR3 in the form of n groups and Z rows and M coefficients in each row; marking the corresponding storage cells in the missing rows as invalid data to fill H rows for splicing can prevent data confusion during storage and reading, and ensure the accuracy of stored and read data; the above H is equal to the ratio of the word length of DDR3 storage unit to the word length of NOR flash storage unit; see Figure 2In this embodiment, the word length of the NOR flash storage unit is 32 bits, and the word length of the DDR3 storage unit is 128 bits, then H is equal to 4, Z is 5, and when splicing, each 4 lines are spliced ​​as a group, and each group of 18 lines, each line with 1498 coefficients, is spliced ​​and stored in 5 lines of DDR3, and each line includes 1498 coefficients. In this embodiment, the first storage unit of the first row of DDR3 specifically stores the following contents: the upper 32 bits store the first coefficient of the first row of the first 18 rows of 1498 coefficients in each row stored in the NOR flash; the second upper 32 bits store the first coefficient of the second row of the first 18 rows of 1498 coefficients in each row stored in the NOR flash; the second lower 32 bits store the first coefficient of the third row of the first 18 rows of 1498 coefficients in each row stored in the NOR flash; the lower 32 bits store the first coefficient of the fourth row of the first 18 rows of 1498 coefficients in each row stored in the NOR flash; and so on. In this embodiment, the second lower 32 bits and the lower 32 bits in each storage unit in the fifth row of DDR3 are marked as invalid data due to the lack of coefficients, and the same is true for the second lower 32 bits and the lower 32 bits in the 10th row, ... the 265th (53×5) row;

[0066] 1.3) According to the actual hardware environment of the wave aberration correction system, set the voltage gain matrix in the NOR flash , the above voltage gain matrix The size is ; In this embodiment, the voltage gain matrix The size is ;

[0067] The order of the above steps 1.1), 1.2) and 1.3) can be interchanged, in no particular order;

[0068] Step 2: Read the wavefront image;

[0069] Step 3: See Figure 3 According to the number a of Hartmann's subapertures, the read wavefront image is windowed and packetized to obtain b windowed image packets, each of which contains a / b windowed images, where b is the set number of packets and the value of a / b is an integer. In order to improve the parallel processing speed without wasting too many hardware resources, it is preferred that the set number of packets b should meet the following conditions: the value of a / b can divide the data length D that can be stored in each row in DDR3. In this embodiment, b is set to 7, and a / b is 107. , preferably D is an integer multiple of 107;

[0070] Step 4: For each of the above windowed image packages obtained in step 3, the centroid offset of each windowed image in each package in the X and Y directions is calculated in parallel to obtain a centroid offset matrix , the above centroid offset matrix The size is ; In this embodiment, the centroid offset matrix The size is ;

[0071] Step 5: FPGA reads the above response function coefficient matrix stored in step 1.2) from DDR3 ; Define that each vertical group in DDR3 includes a / b coefficients. When reading, first read the first vertical group of the first group of Z rows, then read the second vertical group of the first group of Z rows, and so on. After reading the first group of Z rows, read the second group of Z rows, and so on, until the above response function coefficient matrix is ​​read. All coefficients in ; see Figure 2 In this embodiment, it is defined that each vertical group in DDR3 includes 107 coefficients, so the response function coefficient matrix In DDR3, there are 14 vertical groups stored in total. The reading method is to start from the first row and read the first vertical group of rows 1 to 5 in sequence. After completion, start from the first row and read the second vertical group of rows 1 to 5 in sequence. After reading the first group of 5 rows, read the second group of 5 rows in sequence until the above response function coefficient matrix is ​​read. And every time a vertical group in a group of Z rows is read, the centroid offset matrix is ​​immediately obtained. and the response function coefficient matrix The product matrix , and the centroid offset matrix obtained in step 4 In this embodiment, each time a vertical group of 5 rows is read, the operation is performed immediately, and the product matrix is ​​finally calculated. ; Product matrix The size is ;

[0072] Step 6: Set the initial matrix of wave aberration control voltages according to step 1.1) , the voltage gain matrix set in step 1.3) And the product matrix calculated in step 5 , using formula [1], calculate the wave aberration control voltage matrix , formula [1] is:

[0073]

[0074] In formula [1]: The wave aberration control voltage matrix is ​​represented as In this embodiment, Matrix of It represents the wave aberration control voltage matrix obtained by the previous calculation adjacent to this one. When this is the first calculation, Take the initial matrix of wave aberration control voltage set in step 1.1) ; Represents the product matrix With the voltage gain matrix The Hadamard;

[0075] Step 7: Based on the wave aberration control voltage matrix calculated in step 6 , adjusting the surface shape of the deformable mirror in the optical system, and then obtaining a wavefront image after the deformable mirror surface shape is adjusted;

[0076] Step 8: The FPGA executes the program to determine whether the set wave aberration correction end instruction is received; if not, it returns to step 2; if so, the wave aberration correction is ended and the image calculation processing is completed.

[0077] In summary, the present invention is based on the image fast calculation processing method of wave aberration correction, by The response function coefficient matrix Stored in DDR3 in a specific way, it not only improves the response function coefficient matrix The storage speed in DDR3 also lays the foundation for improving the speed of matrix operation in the subsequent calculation of wave aberration control voltage matrix; the response function coefficient matrix Store in DDR3 in a specific way, and then read the response function coefficient matrix from DDR3 in a specific way in combination with the subsequent FPGA , which improves the speed of matrix operations in the process of calculating the wave aberration control voltage matrix; in addition, in calculating the center of mass offset matrix Before reading, the wavefront image is windowed and packaged, which also improves the calculation of the centroid offset matrix. The calculation speed is shown by subsequent experiments. The fast calculation and processing method of the image based on wave aberration correction of the present invention is used to shorten the time of a single calculation of the wave aberration control voltage matrix from the original 160ms to 10.3ms. Figure 3; The frame rate is also increased to 100 frames per second, greatly improving the efficiency of image calculation and processing. The image rapid calculation and processing method based on wave aberration correction of the present invention completes the image rapid calculation and processing based on wave aberration correction through the cooperation of three hardwares: NORflash, DDR3, and FPGA. It has fast storage, reading and calculation speeds, good real-time performance, low power consumption, and high reliability. It provides technical support for on-board wave aberration correction and also provides a reference for on-board image rapid calculation and processing in other fields. The present invention is suitable for image processing of Hartmann wavefront correction.

Claims

1. A method for rapid image calculation and processing based on wave aberration correction, the method is implemented based on a wave aberration correction system, characterized in that: The following steps are involved: Step 1: Initialize the parameters of the wave aberration correction control system, including: 1.1) According to the number of correction voltage loops N and the calibration results of the wave aberration correction system, set the initial matrix of wave aberration control voltage in NOR flash , the wave aberration control voltage initial matrix for Matrix of 1.2) According to the number of Hartmann subapertures a and the number of correction voltage loops N, the response function coefficient matrix of the center of mass offset and the wave aberration control voltage adjustment is set: , the response function coefficient matrix The size is , ; Then split each C column into n groups The parallel coefficient matrix, ; Then, in parallel, the coefficients with the same row number in each H column of each parallel coefficient matrix are concatenated and stored one by one in the corresponding storage unit in a row of DDR3, and stored in DDR3 in the form of n groups, each group with Z rows and each row storing M coefficients; 2≤H≤C; Z is the value obtained by rounding up the quotient of C divided by H; 1.3) According to the actual hardware environment of the wave aberration correction system, set the voltage gain matrix in the NOR flash , the voltage gain matrix The size is ; Step 2: Read the wavefront image; Step 3: According to the number a of Hartmann's sub-apertures, the read wavefront image is windowed and packaged to obtain b windowed image packages, each of which contains a / b windowed images, where b is the set number of packages and the value of a / b is an integer; Step 4: For each of the windowed image packages obtained in step 3, the centroid offsets of each windowed image in each package in the X and Y directions are calculated in parallel to obtain a centroid offset matrix , the centroid offset matrix The size is ; Step 5: FPGA reads the response function coefficient matrix stored in step 1.2) from DDR3 ; Define that each vertical group in DDR3 includes a / b coefficients. When reading, first read the first vertical group of the first group of Z rows, then read the second vertical group of the first group of Z rows, and so on. After reading the first group of Z rows, read the second group of Z rows, and so on, until the response function coefficient matrix is ​​read. All coefficients in ; and each time a vertical group in a group of Z rows is read, it is immediately based on the centroid offset matrix and the response function coefficient matrix The product matrix , and the centroid offset matrix obtained in step 4 The corresponding operations are performed on the elements at the corresponding positions in the matrix, and the product matrix is ​​finally calculated. ; Step 6: Set the initial matrix of wave aberration control voltages according to step 1.1) , the voltage gain matrix set in step 1.3) And the product matrix calculated in step 5 , using formula [1], calculate the wave aberration control voltage matrix , formula [1] is: ； In formula [1]: The wave aberration control voltage matrix is ​​represented as Matrix of It represents the wave aberration control voltage matrix obtained by the previous calculation adjacent to this one. When this is the first calculation, Take the initial matrix of wave aberration control voltage set in step 1.1) ; Represents the product matrix With the voltage gain matrix The Hadamard; Step 7: Based on the wave aberration control voltage matrix calculated in step 6 , adjusting the surface shape of the deformable mirror in the optical system, and then obtaining a wavefront image after the deformable mirror surface shape is adjusted; Step 8: The FPGA executes the program to determine whether the set wave aberration correction end instruction is received; if not, it returns to step 2; if so, the wave aberration correction is ended and the image calculation processing is completed.

2. The image fast calculation and processing method based on wave aberration correction according to claim 1, characterized in that: The wave aberration correction system includes a wavefront sensor, a wave aberration correction control system, N correction voltage circuits and an optical system which are connected in sequence to form a circuit; the wavefront sensor includes a plurality of Hartmann sub-apertures; the wavefront sensor is used to obtain the wavefront image generated by the optical system and transmit it to the wave aberration correction control system; the wave aberration correction control system is used to calculate and adjust the wave aberration control voltage in each correction voltage circuit according to the transmitted wavefront image, so that the surface shape of the deformable mirror in the optical system is changed to realize the correction of the wave aberration and generate a wavefront image after the deformable mirror surface shape is adjusted.

3. The image rapid calculation and processing method based on wave aberration correction according to claim 1, characterized in that: In step 1.2), the coefficients with the same row number in each H column of each parallel coefficient matrix are spliced ​​in parallel and stored one by one in the corresponding storage unit in a row of DDR3. When the coefficients are stored in DDR3 in the form of n groups with Z rows in each group and M coefficients in each row, the parallel coefficient matrices are first stored in parallel in NOR flash, and then spliced ​​in parallel from NOR flash and stored in DDR3; The H is equal to the ratio of the word length of the DDR3 storage unit to the word length of the NOR flash storage unit.

4. The image fast calculation and processing method based on wave aberration correction according to claim 3 is characterized in that: In step 1.2), the parallel coefficient matrices are first stored in parallel in the NOR flash, and then spliced ​​and stored in parallel in the DDR3 from the NOR flash, specifically: First, store the coefficients of a column in the parallel coefficient matrix into a row in the NOR flash. The coefficients of each column in the parallel coefficient matrix are stored one by one in N rows in the NOR flash in sequence, and are stored in the NOR flash in the form of n groups, each group has C rows, and each row has M coefficients; Then, in parallel, each group of C rows of M coefficients stored in the NOR flash are spliced ​​into one row of DDR3 in the manner of splicing every H rows of coefficients. The coefficients stored in the storage cells with the same column order in each H rows are spliced ​​in sequence from high to low bits in the corresponding storage cells in a row of DDR3 according to the row number. Moreover, when each group of C rows of M coefficients is spliced ​​as a group with every H rows of coefficients, if the last group has less than H rows, the corresponding storage cells in the missing rows are marked as invalid data to fill up H rows, and then spliced. Finally, it is stored in DDR3 in the form of n groups of Z rows with M coefficients stored in each row.

5. The image fast calculation and processing method based on wave aberration correction according to claim 4 is characterized in that: The NOR flash storage unit word length is 32 bits, the DDR3 storage unit word length is 128 bits, and in step 1.2), H is equal to 4.

6. The image fast calculation and processing method based on wave aberration correction according to claim 1, characterized in that: In step 3, the setting of the number of subpackets b should meet the following conditions: the value of a / b can evenly divide the data length D that can be stored in each row in DDR3.

Images