Centroid processing method, high-speed wavefront sensor, equipment and medium

Abstract

The invention discloses a centroid processing method, a high-speed wavefront sensor, equipment and a medium, and the method comprises the steps: obtaining original image data obtained by an area array detector, carrying out the centroid extraction of a plurality of light spot centroids, obtaining the pixel data of the area array detector, reading the pixel data of the area array detector according to the line, and obtaining the data of the high-speed wavefront sensor. The method comprises the steps of performing storage and centroid calculation through at least one first cache group to obtain a plurality of centroid coordinates, reading calibration coordinates in a first memory, performing calculation on the plurality of centroid coordinates and the calibration coordinates to obtain a plurality of centroid position deviation values, and packaging the plurality of centroid position deviation values to obtain wavefront information. The FPGA-based multi-centroid parallel high-speed processing method is realized, the real-time performance is high, the dependence on on-chip resources is small, the data output bandwidth is low, the universality is relatively high, the wavefront correction processor resources are released, and the wavefront closed-loop correction bandwidth is improved.

Description

Technical Field

[0001] The present application belongs to the field of adaptive optics technology, and in particular, relates to a centroid processing method based on FPGA. Background Art

[0002] Large-aperture optical telescopes can achieve high-resolution optical imaging and improve optical receiving energy, and have important applications in astronomical observations, space optical communications, and space quantum communication systems. However, due to the presence of atmospheric turbulence, large-aperture optical telescopes are far from their expected performance. Current experiments have shown that large-aperture optical telescopes installed on the surface have a resolution equivalent to that of an optical telescope with a diameter of 100 mm due to the presence of atmospheric turbulence. In order to reduce the impact of atmospheric turbulence, one approach is to select high-altitude areas with low atmospheric turbulence, and the other is to use adaptive optics to perform wavefront correction. Adaptive optics systems dynamically compensate for turbulence, have developed rapidly in recent years, and have been widely used in space optical systems.

[0003] The mainstream adaptive optical system includes wavefront detection, wavefront correction algorithm and deformable mirror. Among them, wavefront detection currently mainly uses SH (Shack-Hartmann) wavefront sensor. Most of the current SH wavefront sensors output raw images and cannot directly obtain wavefront information. The raw image information needs to be output to the back-end server. The server processes the raw image according to relevant auxiliary information and finally inverts the wavefront aberration. High frame rate wavefront image processing occupies processor resources. To achieve high wavefront correction bandwidth, high performance requirements are placed on the processing server. Therefore, wavefront sensors need to be improved and enhanced. Summary of the invention The purpose of the present application is to provide a centroid processing method, a high-speed wavefront sensor, a device and a medium, which can solve at least one of the above problems.

[0004] In a first aspect, a centroid processing method is provided. The high-speed wavefront sensor includes: an array detector, a first cache group, and a first memory. The steps include: Acquire raw image data obtained by the area array detector, wherein the raw image data includes a plurality of light spot centroids; Extract the centroids of multiple light spots to obtain pixel data of the array detector; Reading pixel data of the area array detector row by row, storing and calculating the centroid through at least one first buffer group, and obtaining a plurality of centroid coordinates; Reading the calibration coordinates in the first memory, calculating the multiple centroid coordinates with the calibration coordinates respectively, and obtaining multiple centroid position deviation values; Multiple centroid position deviation values ​​are packaged to obtain wavefront information.

[0005] In some embodiments, the step of reading pixel data of the area array detector row by row, storing and calculating the centroid through at least one first cache group, and obtaining a plurality of centroid coordinates specifically includes: The first cache group includes a first cache and a second cache; Reading pixel data of the area array detector row by row to obtain a plurality of pixel rows, and calculating the number of write cycles according to the plurality of pixel rows; Multiple pixel rows are cyclically written into the first buffer and the second buffer according to the number of write cycles, and the first buffer and the second buffer are used to calculate the centroid of the written pixel rows respectively to obtain multiple centroid coordinates. Thus, by providing the first buffer and the second buffer, the two groups of buffers are operated to improve the real-time performance of the system.

[0006] In some embodiments, the step of cyclically writing the plurality of pixel rows into the first buffer and the second buffer according to the number of write cycles, and performing centroid calculation on the written pixel rows respectively through the first buffer and the second buffer to obtain a plurality of centroid coordinates, specifically includes: The plurality of pixel rows are cyclically written into the first buffer and the second buffer according to the number of write cycles; and the plurality of centroids in the written pixel rows are respectively parallelly calculated by the first buffer and the second buffer to obtain a plurality of centroid coordinates. Thus, the parallel calculation can improve the calculation speed and ensure that the time consumed for a group of centroid calculations does not exceed the time consumed for a group of image outputs.

[0007] In some implementations, the reading of the calibration coordinates in the first memory and calculating the multiple centroid coordinates with the calibration coordinates to obtain multiple centroid position deviation values ​​specifically include: If the external light source is a standard light source, it is judged that the current environment is under the condition of no atmospheric turbulence; If the front environment is in a condition without atmospheric turbulence disturbance, the original image data obtained by the array detector is obtained; Read the original image data to calculate the centroid and obtain the calibration coordinates; The obtained calibration coordinates are written into the first memory. Thus, the calibration coordinates can be stored in the memory in advance, so that they can be easily retrieved and used at any time.

[0008] In some implementations, the step of packaging a plurality of centroid position deviation values ​​to obtain wavefront information specifically includes: A plurality of centroid position deviation values ​​are packaged in a preset frame format to obtain wavefront information.

[0009] In a second aspect, a high-speed wavefront sensor is provided, which includes: a control device and a planar array detector, a microlens array, a first cache group and a first memory electrically connected to the control device, and the control device is used to execute the centroid processing method described above.

[0010] In some implementations, the first cache group includes an even number of on-chip cache BRAMs or off-chip dynamic memories DDRs, thereby maximally reducing the occupation of FPGA resources.

[0011] In some embodiments, an output interface is also included, the control device is electrically connected to the output interface, and the output interface is one of 100M Ethernet, 1000M Ethernet or USB 3.0. Thus, the processed data can be output, and the interface can also be compatible with different aperture numbers.

[0012] According to a third aspect, a wavefront processing device is provided, comprising: a memory and at least one processor, wherein instructions are stored in the memory; at least one of the processors calls the instructions in the memory so that the high-speed wavefront processing device executes the various steps of the centroid processing method described above.

[0013] In a fourth aspect, a computer-readable storage medium is provided, on which instructions are stored, and when the instructions are executed by a processor, the various steps of the centroid processing method described above are implemented.

[0014] The present application has at least the following beneficial effects: through a centroid processing method and related devices, a multi-centroid parallel high-speed processing method based on FPGA is realized, which has high real-time performance, low dependence on on-chip resources, low data output bandwidth, high universality, releases wavefront correction processor (CPU) resources, and improves wavefront closed-loop correction bandwidth. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 This is a schematic diagram of wavefront imaging without center obstruction provided by an embodiment of the present application; Figure 2 This is a schematic diagram of wavefront imaging with central obstruction provided in an embodiment of the present application; Figure 3 This is a schematic diagram of the correspondence between pixels and sub-apertures of the area array detector provided in an embodiment of the present application; Figure 4 It is a schematic diagram of the centroid writing and calculation process provided in an embodiment of the present application; Figure 5 is a structural diagram of a wavefront processing device provided in an embodiment of the present application; Figure 6 It is a schematic diagram of the process provided by the embodiment of the present application; Figure 7 It is a system block diagram of a high-speed wavefront processor provided in an embodiment of the present application. DETAILED DESCRIPTION

[0016] The following description provides specific application scenarios and requirements of this specification, with the purpose of enabling those skilled in the art to make and use the contents of this specification. Various local modifications to the disclosed embodiments will be apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments and applications without departing from the spirit and scope of this specification. Therefore, this specification is not limited to the embodiments shown, but to the widest scope consistent with the claims.

[0017] It should be noted that the embodiments and features in the embodiments of the present invention can be combined with each other when there is no conflict in structure or function. The present application is further described in detail below in conjunction with the accompanying drawings.

[0018] Most of the current SH wavefront sensors output raw images and cannot directly obtain wavefront information. The raw image information is output to the backend server. The server processes the raw image based on relevant auxiliary information to obtain the centroid information of the light spot corresponding to each sub-aperture, and then calculates the centroid error, and then inverts to obtain the wavefront aberration. This method has the following disadvantages: First, the Greenwood frequency caused by atmospheric turbulence is usually above KHz, which requires that the frame rate of the wavefront sensor should not be less than this frequency. High frame rate images result in a huge amount of data output by the detector. At present, the output interface of KHz frame rate wavefront sensors abroad is usually 10 Gigabit Ethernet or CoaXPress serial interface. These two interfaces have high requirements on the hardware resources of the wavefront processing computer and are difficult to develop. Secondly, high frame rate wavefront image processing occupies processor resources. To achieve high wavefront correction bandwidth, high performance requirements are placed on the processing server.

[0019] In order to overcome the above problems, the present application provides a centroid processing method. The high-speed wavefront sensor includes: an array detector 503, a first cache group 502, and a first memory 505. Figure 6 , the specific steps include: S100, acquiring original image data obtained by the area array detector, wherein the original image data includes a plurality of light spot centroids; S200, extracting the centroids of multiple light spots to obtain pixel data of the area array detector; S300, reading pixel data of the area array detector row by row, storing and calculating the centroid through at least one first cache group, and obtaining a plurality of centroid coordinates; S400, read the calibration coordinates in the first memory, calculate the multiple centroid coordinates with the calibration coordinates respectively, and obtain multiple centroid position deviation values; wherein, the centroid position deviation value is the error value between the calibration centroid position after centroid extraction, and thus can be obtained by subtracting the centroid coordinates from the calibration coordinates. The calibration coordinates are data pre-stored in the first memory 505 through early calibration, that is, the centroid position of the ideal wavefront of each sub-aperture obtained under the condition of no atmospheric turbulence disturbance. For example, it can be pre-stored in the non-volatile memory flash, and after the system is powered on, the calibration data in the flash can be pre-loaded into the FPGA processor.

[0020] S500, multiple centroid position deviation values ​​are packaged to obtain wavefront information. Preferably, multiple centroid position deviation values ​​can be packaged in a preset frame format to obtain wavefront information. The frame format and the packaging method can be set according to actual use needs and are not limited here.

[0021] Reference Figure 1 and Figure 2 , which is a schematic diagram of wavefront imaging on the detector of two typical optical systems. Common large-aperture space optical telescopes are mainly reflective structures, including coaxial reflective structures and off-axis reflective structures. The coaxial structure has good symmetry and is simpler to implement in engineering, but there is central obstruction and a small field of view. Figure 2 As shown. The off-axis system has a larger field of view and no central obstruction. Figure 1 As shown, the engineering implementation is relatively complex. Figure 1 and Figure 2 Each sub-aperture in the image subdivides the wavefront and focuses it on the detector through a microlens. In other words, the original image data obtained by the wavefront sensor is a light spot matrix, which involves the extraction of hundreds of light spot centroids. However, the existing multi-centroid extraction algorithms based on FPGA processors are mostly oriented to random light spot arrangements, and their processing flow is relatively complex and the real-time performance is not high.

[0022] Specifically, refer to Figure 3 , determine the corresponding relationship between the position of the pixel and the sub-aperture on the detector according to the sub-aperture size. Assuming that the sub-aperture size is a*a and the size of each pixel is d*d, the number of pixels n occupied by each sub-aperture is (a / d)2, as Figure 3As shown, there are M columns of subapertures, each subaperture contains n pixels, and the total number of pixels is M*n. The centroid is extracted within the n*n pixel range of each subaperture. After the centroid is extracted for each individual subaperture, a centroid weighted algorithm can be used for calculation. The centroid coordinates calculated in real time are subtracted from the calibration coordinates pre-stored in the first buffer 505, for example, using the non-volatile memory flash, and the centroid position deviation value of each subaperture is calculated. The centroid position deviation values ​​are packaged to obtain wavefront information. It should be noted that the centroid weighted calculation can use the existing centroid weighted algorithm for calculation, which is not an improvement of the present application and will not be repeated here.

[0023] Further, the first cache group 502 includes a first cache and a second cache; the step of reading pixel data of the area array detector 503 row by row, storing and calculating the centroid through at least one first cache group 502, and obtaining multiple centroid coordinates specifically includes: Reading pixel data of the area array detector 503 row by row to obtain a plurality of pixel rows, and calculating the number of write cycles according to the plurality of pixel rows; The plurality of pixel rows are cyclically written into the first buffer and the second buffer according to the writing cycle number, and the centroids of the written pixel rows are calculated respectively by the first buffer and the second buffer to obtain a plurality of centroid coordinates.

[0024] Specifically, in order to improve the real-time performance of the system, the first cache group 502 includes a first cache and a second cache by utilizing the characteristic that the pixels of the area array detector 503 are read out line by line in units of lines. For example, the first buffer and the second cache use two on-chip cache BRAMs, one of which is used as a ping BRAM and the other as a pong BRAM, and each on-chip cache stores pixel data and performs a ping-pong operation. In this way, the centroid of the light spot is stored in one on-chip cache BRAM, and the centroid can be calculated at the same time in another on-chip cache BRAM, so as to achieve parallel processing without affecting each other.

[0025] like Figure 4As shown, in order to realize parallel processing of the centroids of the light spots of the M sub-apertures in each cache BRAM, each group of cache BRAMs contains M independent storage units. Considering that the image of the area array detector 503 is output line by line, it can be written into a group of n lines, for example, when 2 lines of pixel rows are obtained, 1 line is a group, and written into the ping-pong BRAM. When a group of original images is full, the BRAM is switched, and the image data that will be read is written into the pong BRAM. At the same time, the centroid weighted average calculation is performed on the M sub-aperture images stored in the ping-pong BRAM, and the centroid result is calculated and output. Preferably, M independent centroid calculation units are used to perform parallel calculation on a group of centroids, that is, parallel calculation can also be used for the M sub-aperture images in a group, so as to improve the calculation speed and ensure that the time consumed for calculating a group of centroids does not exceed the time for outputting a group of images.

[0026] Similarly, when the pong BRAM is full, the centroid of the image data in the pong BRAM is calculated, and the image data is switched to the ping BRAM for storage. That is to say, the cycle will continue to write the read image data to the ping BRAM or pong BRAM, and the other full BRAM can be calculated. Relying on this mechanism, the centroid of the spot of M sub-apertures in a row can be calculated in real time. This method can minimize the occupation of FPGA resources, and the parallel operation is at most M centroid calculation units working at the same time. Therefore, by using two ping-pong caches, a centroid extraction process that matches the pixel readout rate of the detector can be achieved, while reducing the dependence on the FPGA on-chip cache.

[0027] Preferably, before step S400, the method further includes: If the external light source is a standard light source, it is judged that the current environment is under the condition of no atmospheric turbulence; If the current environment is free of atmospheric turbulence disturbance, the original image data obtained by the array detector is obtained; Read the original image data to calculate the centroid and obtain the calibration coordinates; The obtained calibration coordinates are written into the first memory.

[0028] In this embodiment, the calibration coordinates can be obtained in the same process as the center of mass coordinates, for example, reading and writing and center of mass calculation are performed respectively through two cache BRAMs, except that the external wavefront corresponding to the calibration coordinates is a standard light source, that is, the center of mass coordinates under conditions without atmospheric turbulence disturbance, while the actual application is the center of mass coordinates in a real environment.

[0029] Therefore, the present invention extracts multiple centroids for the spot matrix. According to its arrangement rules, the parallel processing mechanism of FPGA can be maximized. Combined with the serial readout mechanism of the detector, real-time calculation of the centroid of the spot can be realized. The centroid coordinate calculation and deviation value calculation all adopt pipeline operation, with low latency, no system resource occupation, and great engineering feasibility. Compared with the existing centroid processing method, the present application realizes a multi-centroid parallel high-speed processing method based on FPGA, which has high real-time performance, low dependence on on-chip resources, low data output bandwidth, and great universality, releasing the resources of the wavefront correction processor (CPU) and improving the wavefront closed-loop correction bandwidth.

[0030] In a second aspect, the present application provides a high-speed wavefront sensor, referring to Figure 7 The high-speed wavefront sensor includes: a control device 501 and a planar array detector 503, a microlens array 504, a first cache group 502 and a first memory 505 electrically connected to the control device. The control device 501 is used to execute the centroid processing method in any optional implementation method of the above embodiments.

[0031] Specifically, the area array detector 503 is used for real-time capture of the wavefront; the microlens array 504 is used to focus the image on the area array detector 503; when used specifically, the specifications of the area array detector 503 can be determined by the number of microlenses and the size of the sub-aperture. In order to ensure real-time capture of the wavefront, the area array detector 503 adopts a high frame rate global exposure mode. The number of first cache groups 502 can be an even number, such as two groups of on-chip cache BRAM or off-chip dynamic storage DDR, which are used for real-time storage and reading of image data; the first memory 505 can be a non-volatile memory Flash, which can still store relevant data after power failure, ensuring that the relevant data can be obtained at any time when needed after power is turned on, and is used to store the various centroid positions of calibration; In actual use, an output interface 506 can also be set. The control device 501 is electrically connected to the output interface 506 to output wavefront information, that is, the packaged center of mass position deviation value, which is the error value between the center of mass position after extraction and the calibrated center of mass position. The data volume is relatively small. For example, for a wavefront sensor with 100 sub-apertures, the frame rate is 1KHz, and each center of mass error is represented by two 16 bits. The total data volume is 3.2Mbps, which can be achieved by using an ordinary 100M network. In order to be compatible with more aperture numbers, a conventional Gigabit network or USB3.0 interface can be used.

[0032] Therefore, the high-speed wavefront sensor in this application implements the wavefront raw image processing on the wavefront sensor, processes the raw image, obtains the centroid data, and then outputs it, which can greatly reduce the data bandwidth, increase the closed-loop bandwidth of the wavefront correction system, and realize the compensation of atmospheric turbulence. It has the function of real-time centroid extraction and can be applied to large-aperture space optical application scenarios.

[0033] Thirdly, Figure 5 6 is a schematic diagram of the structure of a wavefront processing device provided by an embodiment of the present invention. The wavefront processing device 600 may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPU) 610 (for example, one or more processors) and a memory 620, and one or more storage media 630 (for example, one or more mass storage devices) storing application programs 633 or data 632. Among them, the memory 620 and the storage medium 630 may be short-term storage or permanent storage. The program stored in the storage medium 630 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the wavefront processing device 600.

[0034] Furthermore, the processor 610 may be configured to communicate with the storage medium 630 and execute a series of instruction operations in the storage medium 630 on the wavefront processing device 600 to implement the steps of the wavefront processing method provided in the above-mentioned method embodiments.

[0035] The wavefront processing device 600 may further include one or more power supplies 640, one or more wired or wireless network interfaces 650, one or more input and output interfaces 660, and / or one or more operating systems 631, such as Windows Serve, Mac OS X, Unix, Linux, FreeBSD, etc. It will be appreciated by those skilled in the art that Figure 5 The structure of the wavefront processing device shown does not constitute a limitation of the wavefront processing device, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0036] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. Instructions are stored in the computer-readable storage medium. When the instructions are executed on a computer, the computer executes the steps of the centroid processing method in the embodiment.

[0037] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described system, device, or unit can refer to the corresponding process in the aforementioned method embodiment and will not be repeated here.

[0038] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the whole or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0039] The above are only preferred implementations of the present application. It should be pointed out that, for ordinary technicians in this field, several modifications and improvements can be made without departing from the creative concept of the present application, which all belong to the protection scope of the present application.

Claims

1. A centroid processing method, characterized in that: The high-speed wavefront sensor comprises: an array detector, a first buffer group, and a first memory, and includes the steps of: Acquire raw image data obtained by the area array detector, wherein the raw image data includes a plurality of light spot centroids; Extract the centroids of multiple light spots to obtain pixel data of the array detector; Reading pixel data of the area array detector row by row, storing and calculating the centroid through at least one first buffer group, and obtaining a plurality of centroid coordinates; Reading the calibration coordinates in the first memory, calculating the multiple centroid coordinates with the calibration coordinates respectively, and obtaining multiple centroid position deviation values; Multiple centroid position deviation values ​​are packaged to obtain wavefront information.

2. A centroid processing method according to claim 1, characterized in that: The step of reading the pixel data of the area array detector row by row, storing and calculating the centroid through at least one first cache group, and obtaining a plurality of centroid coordinates specifically includes: The first cache group includes a first cache and a second cache; Reading pixel data of the area array detector row by row to obtain a plurality of pixel rows, and calculating the number of write cycles according to the plurality of pixel rows; The plurality of pixel rows are cyclically written into the first buffer and the second buffer according to the writing cycle number, and the centroids of the written pixel rows are calculated respectively by the first buffer and the second buffer to obtain a plurality of centroid coordinates.

3. A centroid processing method according to claim 2, characterized in that: The step of writing the plurality of pixel rows into the first buffer and the second buffer in a loop according to the number of write cycles, and performing centroid calculation on the written pixel rows respectively through the first buffer and the second buffer to obtain a plurality of centroid coordinates, specifically includes: cyclically writing the plurality of pixel rows into the first buffer and the second buffer according to the number of write cycles; The first buffer and the second buffer are used to respectively perform parallel calculations on the multiple centroids in the written pixel row to obtain multiple centroid coordinates.

4. A centroid processing method according to claim 1, characterized in that: The step of reading the calibration coordinates in the first memory and calculating the plurality of centroid coordinates with the calibration coordinates respectively to obtain a plurality of centroid position deviation values ​​also includes: Get the type of external light source. If the external light source is a standard light source, determine whether the current environment is in a condition without atmospheric turbulence. If the current environment is in a condition without atmospheric turbulence disturbance, the original image data obtained by the array detector is obtained; Read the original image data to calculate the centroid and obtain the calibration coordinates; The obtained calibration coordinates are written into the first memory.

5. A centroid processing method according to claim 1, characterized in that: Multiple centroid position deviation values ​​are packaged to obtain wavefront information, including: A plurality of centroid position deviation values ​​are packaged in a preset frame format to obtain wavefront information.

6. A high-speed wavefront sensor, comprising: A control device and a planar array detector, a microlens array, a first cache group and a first memory electrically connected to the control device, wherein the control device is used to execute the centroid processing method as described in any one of claims 1 to 5.

7. A high-speed wavefront sensor according to claim 6, characterized in that: The first cache group includes an even number of on-chip cache BRAMs or off-chip dynamic memories DDRs.

8. A high-speed wavefront sensor according to claim 6, characterized in that: It also includes an output interface, the control device is electrically connected to the output interface, and the output interface is one of 100M Ethernet, 1000M Ethernet or USB 3.

0.

9. A wavefront processing device, characterized in that: The wavefront processing device comprises: a memory and at least one processor, wherein instructions are stored in the memory; At least one of the processors calls the instructions in the memory to enable the high-speed wavefront processing device to perform each step of the centroid processing method as described in any one of claims 1-5.

10. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instructions are executed by a processor, the various steps of the centroid processing method as described in any one of claims 1 to 5 are implemented.

Images