FPGA-based astronomical image astronomical image detection and sub-pixel centering method and system

Abstract

The invention discloses an astronomical image astronomical image detection and sub-pixel centering method and system based on an FPGA, and the method comprises the steps: carrying out the binarization of pixels according to an input gray image and a threshold array, carrying out the single scanning of the binarized pixels, carrying out the detection of a connected domain, eliminating the connected domain with a conflict in a label, and updating the astronomical image parameters; storing and recording corresponding astronomical image parameters during detection, and outputting the astronomical image parameters when the astronomical image is completely detected; based on the astronomical image parameters, calculating an astronomical image mass center with sub-pixel precision through a two-dimensional correction distance method, and outputting a final detection result; according to the astronomical observation image astronomical image detection method, assembly line type real-time astronomical image detection can be achieved, astronomical observation image astronomical image detection efficiency and processing capacity are improved, the data caching amount can be greatly reduced, FPGA storage resource occupation is greatly reduced, the capacity of processing larger-size images is achieved, and the upper limit of the size of the processable image can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of astronomical image processing, and in particular to an astronomical image star detection and sub-pixel centering method and system based on FPGA. Background Art

[0002] The traditional processing flow of astronomical observation images is that the camera collects images and transmits them to the host computer through cables for processing, or transmits them to the data processing center through the network for unified processing. This process requires the transmission of a large amount of data. However, for astronomical measurement, the only valid information is the position, shape and flow information of the stars. In addition, there is a high latency, and the real-time performance cannot meet the requirements for task scenarios such as telescope star guidance calculation.

[0003] Compared with graphics processing units (GPUs) and central processing units (CPUs), field programmable gate arrays (FPGAs) have higher energy efficiency and better flexibility than application-specific integrated circuits (ASICs), and can effectively accelerate astronomical image processing.

[0004] Traditional 8-neighborhood and 4-neighborhood detection methods require multiple scans and cannot effectively use FPGA features to accelerate image processing; other point spread function (PSF)-based methods have high centering accuracy but require complex calculations and multiple iterations, and cannot effectively use FPGA to accelerate processing. Therefore, FPGA-based astronomical image star detection and sub-pixel centering methods and systems are urgently needed to solve the above problems. Summary of the invention

[0005] The purpose of the present invention is to provide an astronomical image star detection and sub-pixel centering method and system based on FPGA, which can realize single-scan pixel, pipelined astronomical image star detection and sub-pixel centering, make full use of the FPGA parallel processing advantages, accelerate the processing flow of astronomical observation images, improve processing capabilities, and reduce useless data transmission.

[0006] In order to solve the above technical problems, the present invention adopts the following technical solution: an astronomical image star detection and sub-pixel centering method based on FPGA, comprising the following steps:

[0007] S1, binarize the pixels according to the input grayscale image and threshold array;

[0008] S2, a single scan of the binarized pixels, to detect connected domains and eliminate connected domains with label conflicts, and to update the constellation parameters;

[0009] S3, saving and recording the corresponding star parameters during the detection period, and outputting the star parameters when the star is completely detected;

[0010] S4. Based on the star parameters, the star centroid with sub-pixel accuracy is calculated through the two-dimensional corrected distance method, and the final detection result is output.

[0011] Preferably, in step S1, the multi-pixel parallel input is converted into a single-pixel serial output, and the row and column numbers of the pixels are output synchronously, and the serially transmitted pixels are binarized:

[0012]

[0013] Among them, v (m,n) Represents the grayscale value of the input image m rows and n columns, T (p,q) represents the value of the pth row and qth column of the threshold array, u (m,n) Represents the gray value of the mth row and nth column of the binarized image, and p,q and m,n satisfy the following conditions:

[0014]

[0015] in, It means rounding down. M and N are divisible by P and Q respectively. When P=1 and Q=1, it is equivalent to using the global threshold.

[0016] Preferably, in step S2, each pixel is scanned row by row from top to bottom, neighborhood detection is performed to generate star labels, and corresponding star parameters are recorded at the same time.

[0017] Preferably, the neighborhood detection is specifically as follows:

[0018] Update the pixel star image label record of the previous row. If the current row to be processed is the first row, initialize it to unassigned label;

[0019] Read the pixels in sequence, and when the value of the pixel is not zero, read the star labels of the pixels to the upper right, above, upper left, and left of the pixel, and check in a counterclockwise direction whether the pixels to the upper right, above, upper left, and left of the pixel are allocated. Based on the detection results, perform the following:

[0020] When the constellation labels of adjacent pixels have not been assigned, a new constellation label is assigned to the pixel;

[0021] When the upper right pixel is not assigned, the upper pixel, the upper left pixel and the left pixel are detected in sequence, and when any pixel is assigned, the pixel is newly assigned the star sign corresponding to the assigned pixel;

[0022] When allocating the upper right pixel, the pixel is newly allocated the same constellation label as the upper right pixel;

[0023] Preferably, unassigned labeled pixels are set to 0.

[0024] Preferably, in the case of upper right pixel allocation, the following is performed for label conflicts:

[0025] When allocating the upper left pixel, determine whether the upper left pixel star sign is equal to the upper right pixel star sign, and if they are not equal, merge the upper left pixel star sign into the upper right pixel star sign;

[0026] When allocating the left pixel, determine whether the left pixel star sign is equal to the upper right pixel star sign, and if they are not equal, merge the left pixel star sign into the upper right pixel star sign.

[0027] Preferably, in step S3, the star parameter records are scanned, and when the coordinate difference between the row being detected and the row where the lowest pixel contained in the star is located is greater than 1, the star is completely detected and the recorded star parameters are output.

[0028] Preferably, in step S4, the centroid of the star image with sub-pixel accuracy is calculated as:

[0029]

[0030] Among them, u (m,n) is the binary grayscale value at (m,n), x m ,y n is the pixel coordinate value at (m,n), and the value range of m and n is the upper, lower, left and right boundary limit area of ​​the star image; ∑ m ∑ n (u (m,n) ·x m ),∑ m ∑ n (u (m,n) ·y n ),∑ m ∑ n u (m,n) They are respectively the cumulative sum x0 of the product of the pixel grayscale value and its column coordinates, the cumulative sum y0 of the product of the pixel grayscale value and its row coordinates, and the total grayscale value L.

[0031] The present invention also discloses an astronomical image star detection and sub-pixel centering system based on FPGA, which is used to implement an astronomical image star detection and sub-pixel centering method based on FPGA. The system includes:

[0032] Image input module: receives parallel input of multiple pixels and converts them into single pixel serial output;

[0033] Parameter storage module: stores pixel segmentation threshold array, array size, image background noise, minimum number of pixels of output star image and minimum signal-to-noise ratio; threshold array includes global threshold or block threshold;

[0034] Binarization segmentation module: reads the array size stored in the parameter storage module, reads the corresponding threshold value according to the row and column where the input pixel is located, and binarizes the pixel;

[0035] Star detection module: scans each pixel, detects its neighborhood, and sends star parameter update, merging signals and data to the star parameter cache management module;

[0036] Astrological parameter cache management module: manages astrological parameters and outputs the detected astrological images;

[0037] Sub-pixel centering calculation module: uses the two-dimensional corrected distance method to calculate the centroid position of the star image;

[0038] Detection result output module: reads the configuration parameters of the parameter storage module and outputs the star detection results.

[0039] Preferably, the detection result output module outputs the minimum number of pixels, or calculates the signal-to-noise ratio and outputs the minimum signal-to-noise ratio:

[0040]

[0041] Among them, L is the total gray value, n is the number of pixels in the star image, and σ is the image background noise.

[0042] Beneficial effects: The present invention provides a method and system for star detection and sub-pixel centering of astronomical images based on FPGA, which can realize single-time scanning of pixels, eliminate conflicting labels without multiple scans, and merge conflicting star labels while completing neighborhood connected domain detection; save parameters for sub-pixel centering during neighborhood detection, output star parameters when stars are completely detected, perform sub-pixel centering, and output detection results, thereby realizing pipelined star detection and sub-pixel centering of astronomical images, effectively utilizing the parallel processing advantages of FPGA, improving the processing efficiency of astronomical measurement observation images, and reducing the transmission of redundant data; and the design of a single scan can greatly reduce the amount of data cache, significantly reduce the FPGA storage resource usage, have the ability to process larger-sized images, and can increase the upper limit of the processable image size. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention.

[0044] In the attached picture:

[0045] Figure 1 A result framework diagram of the astronomical image star detection and sub-pixel centering method and system based on FPGA provided by the present invention;

[0046] Figure 2Schematic diagram of adjacent pixel positions and detection order;

[0047] Figure 3 This is the workflow diagram of the 8-neighborhood star image detection module;

[0048] Figure 4 This is a screenshot of a partial area where the image detection results measured by the FPGA system based on the present invention are superimposed with the original image. DETAILED DESCRIPTION

[0049] In order to make the purpose and advantages of the present invention more clearly understood, the present invention is specifically described in conjunction with the following embodiments. It should be understood that the following text is only used to describe the FPGA-based astronomical image star detection and sub-pixel centering method and system or several specific implementations of the present invention, and does not strictly limit the protection scope of the specific request of the present invention.

[0050] Embodiment: Astronomical image star detection and sub-pixel centering system based on FPGA. This embodiment is implemented using the Milink MZU15A development board, which uses the XCZU15EG-FFVB1156AAZ main chip, with 747,000 logic units, 26.2Mb Block RAM, 2538 DSP units, 341,000 lookup tables, and 682,000 triggers. The design was completed on a workstation with an i9-10920XCPU and 126GB main memory, implemented by Vivado 2019.1, and the operating frequency was set to 250MHz. The test camera uses a QHY-4040CMOS camera with a resolution of 4k*4k; the telescope has an aperture of 14cm and a focal length of 15cm; the camera transmits to the embodiment of this patent through an 85MHz Camera Link full interface, using 8-neighborhood star detection;

[0051] Based on the above, reference Figure 1 As shown, it is a structural framework diagram of this embodiment, which includes:

[0052] Image input module: receives parallel input of multiple pixels and converts them into single pixel serial output;

[0053] Parameter input module: receives external input pixel segmentation threshold array, array size, image background noise, and stores them in the parameter storage module;

[0054] Parameter storage module: stores pixel segmentation threshold array, array size, image background noise, minimum number of pixels of output star image and minimum signal-to-noise ratio; threshold array includes global threshold or block threshold; binary segmentation module and detection result output module read pixel segmentation threshold and image background noise from this module respectively;

[0055] Binarization segmentation module: reads the array size stored in the parameter storage module, reads the corresponding threshold value according to the row and column where the input pixel is located, and binarizes the pixel;

[0056] The star detection module, in this embodiment, is an 8-neighborhood star detection module: scans each pixel, detects its neighborhood, and sends star parameter updates, merge signals and data to the star parameter cache management module;

[0057] Astrological parameter cache management module: manages astrological parameters and outputs the detected astrological images;

[0058] Sub-pixel centering calculation module: uses the two-dimensional corrected distance method to calculate the centroid position of the star image;

[0059] Detection result output module: reads the configuration parameters of the parameter storage module and outputs the star detection results.

[0060] The threshold array size can be configured as needed, with a global threshold of 1 row and 1 column or a block threshold. The block threshold supports four modes: 2 rows and 2 columns, 8 rows and 8 columns, 16 rows and 16 columns, and 64 rows and 64 columns.

[0061] The present invention also discloses an astronomical image star detection and sub-pixel centering method based on FPGA, and the specific working process combined with the above system is as follows:

[0062] Step 1: Configure the camera exposure time, gain, offset and other parameters, and pre-take an image as a flat field reference image to calculate the image binary segmentation threshold array and image background noise;

[0063] Step 2: Input pixel segmentation threshold array, array size and image background noise, as well as the minimum number of pixels of output star image, minimum signal-to-noise ratio and other processing configuration parameters are stored in the parameter storage module.

[0064] Step 3: Use the same camera configuration parameters to start multi-frame continuous shooting, and convert multi-pixel parallel input into single-pixel serial output through the image input module. In this embodiment, 4-pixel parallel input is converted into 1-pixel serial output, and the row and column numbers of the pixels are output synchronously;

[0065] Step 4: The binary segmentation module receives the serially transmitted pixels, and pre-reads the pixel segmentation threshold according to the parameter configuration in the parameter storage module, and performs binary processing on the input pixels;

[0066]

[0067] Among them, v (m,n) Represents the grayscale value of the mth row and nth column of the input image, T (m,n) represents the threshold value of the input image at row m and column n, u (m,n)Represents the gray value of the mth row and nth column of the binarized image. Due to the limitation of storage resources in the specific implementation of FPGA, for the image of M rows and N columns, the threshold is calculated separately for the region divided into P rows and Q columns. The calculation formula for binarization is as follows:

[0068]

[0069] Among them, v (m,n) Represents the grayscale value of the input image m rows and n columns, T (p,q) represents the value of the pth row and qth column of the threshold array, u (m,n) Represents the gray value of the mth row and nth column of the binarized image, and p,q and m,n satisfy the following equation:

[0070]

[0071] in, It means rounding down. M and N are divisible by P and Q respectively. When P=1 and Q=1, it is equivalent to using the global threshold.

[0072] Step 5: Scan the binarized pixels once, scan each pixel row by row from top to bottom, perform neighborhood detection to generate star image labels, perform connected domain detection on the binarized pixels, update star image parameters and record the connected domain parameters in the current row; the parameters include the number of pixels, total gray value, the row where the top pixel is located, the row where the bottom pixel is located, the column where the leftmost pixel is located, the column where the rightmost pixel is located, the cumulative sum of the products of the gray value and the pixel row coordinates, and the cumulative sum of the products of the gray value and the pixel row coordinates;

[0073] The detection sequence is as follows Figure 2 As shown, when the current grayscale value is non-zero, the upper right (A), upper (B), upper left (C), and left (P) pixels and their labels are detected in order.

[0074] Specific testing process: Figure 3 As shown: read in pixels in sequence, and when checking that the value of the pixel is not zero, read the astrological labels of the upper right, upper, upper left, and left pixels of the pixel, and check in a counterclockwise manner whether the upper right, upper, upper left, and left pixels of the pixel are allocated, and whether there is a conflict, that is, whether the labels are equal. Based on the detection results, perform the following:

[0075] Determine whether the upper right pixel label is assigned, if yes, go to branch 1; if not, go to branch 2;

[0076] Branch 1: Determine whether the upper left pixel label is assigned. If so and the upper right label is equal to the upper left label, assign the same label as the upper right to the current pixel. If so but the upper right label is not equal to the upper left label, merge the upper left label into the upper right label, and assign the same label as the upper right to the current pixel; if not, further determine whether the left label is assigned. If the left label is not assigned, assign the same label as the upper right to the current pixel. If it is assigned, check whether the upper right label is equal to the left label. If they are equal, assign the same label as the upper right to the current pixel. If they are not equal, merge the left label into the upper right label, and assign the same label as the upper right to the current pixel;

[0077] Branch 2: Determine whether the pixel label above is assigned. If so, assign the same label as above to the current pixel. If not, determine whether the pixel label above the upper left is assigned. If so, assign the same label as above to the current pixel. If not, determine whether the pixel label below the left is assigned. If so, assign the same label as above to the current pixel. If not, assign a new label to the pixel.

[0078] For the first row of pixels, the upper right (A), top (B), and upper left (C) pixels are considered to be 0 values, the upper right (A) and left (P) pixels in the first column are considered to be 0 values, and the left (P) pixels in the last column are considered to be 0 values;

[0079] Step 6: When a new connected domain is detected and a zero-value pixel is encountered, or when the last column is reached, the connected domain parameters including the number of pixels n, the total gray value L, the product y0 of the row number m0 and the total gray value L, the leftmost n0 and rightmost column number n1 of the connected domain, the cumulative sum x0 of the product of the gray value and the column number, the constellation number, and the conflicting constellation number are output to the constellation parameter cache management module;

[0080] Step 7: In the star image cache management module, the connected domain detection parameters sent by the 8-neighborhood star image detection module are received, and new labels are written accordingly, or existing labels are updated, or conflicting labels are merged. At the same time, it is detected whether the difference between the bottom pixel row sequence number of the star image in the cache and the current detection row sequence number is greater than 1. If it is greater than 1, the parameters of the star image are read out;

[0081] Step 8: Sub-pixel centering: Calculate the center of the star using the 2D corrected distance method as shown below:

[0082]

[0083] Among them, u (m,n) is the binary grayscale value at (m,n), x m ,y n is the pixel coordinate value at (m,n), and the value range of m and n is the upper, lower, left, and right boundary limit area of ​​the star image;

[0084] ∑ m ∑ n (u (m,n) ·x m ),∑ m ∑ n (u (m,n) ·y n ),∑ m ∑ n u (m,n) They are the cumulative sum x0 of the product of the pixel gray value and its column coordinate, the cumulative sum y0 of the product of the pixel gray value and its row coordinate, and the total gray value L, which are calculated in the 8-neighborhood star detection module. In the sub-pixel centering calculation module, the star center x and y are calculated using the following formula:

[0085]

[0086]

[0087] In the detection result output module, the configuration information in the parameter storage module is read to output the star image that meets the minimum pixel number requirement, or the star image signal-to-noise ratio is calculated to output the star image that meets the minimum signal-to-noise ratio; the star image signal-to-noise ratio is calculated using the following formula:

[0088]

[0089] Among them, L represents the flow value, that is, the total gray value of all pixels in the star image after deducting the background, n represents the number of pixels in the star image, and σ represents the image background noise.

[0090] After the above system processing, the star detection work can be completed.

[0091] In this embodiment, the clock frequency of the FPGA star detection module is 250MHz, and the camera outputs 4k*4k resolution, 16-bit quantized images through an 85MHz CameraLink full interface. Two timing interrupt signals are set in the system to measure the time used for the system single-frame detection, starting from the first group of pixels entering the image input module to the output of the last detection result. The average detection time is measured to be about 44ms.

[0092] Figure 4 This is a screenshot of a partial area in which the measured image detection results of the FPGA system of the present invention are superimposed with the original image. The background in the figure is the measured star map, the red circle detection result mark, and the center of the circle is the center of the star image obtained by sub-pixel centering. In this embodiment, the parameter configuration outputs a star image with a minimum pixel number greater than 5.

[0093] Based on the above, the present invention has pipeline working capability, can give full play to the advantages of FPGA and improve data processing efficiency.

[0094] In addition, those skilled in the art should understand that the above-mentioned FPGA system configuration method of the present invention can be implemented using languages ​​such as Verilog, C, C++, etc.; each module can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices. Therefore, the present invention is not limited to any specific combination of hardware and software.

[0095] The embodiments of the present invention are described in detail above in conjunction with the accompanying drawings, but the present invention is not limited to the above embodiments. For ordinary technicians in this technical field, after knowing the contents recorded in the present invention, they can make several equivalent changes and substitutions without departing from the principle of the present invention. These equivalent changes and substitutions should also be regarded as belonging to the protection scope of the present invention.

Claims

1. Astronomical image star detection and sub-pixel centering method based on FPGA, characterized by: The steps include: S1, binarize the pixels according to the input grayscale image and threshold array; S2, a single scan of the binarized pixels, to detect connected domains and eliminate connected domains with label conflicts, and to update the constellation parameters; S3, saving and recording the corresponding star parameters during the detection period, and outputting the star parameters when the star is completely detected; S4. Based on the star parameters, the star centroid with sub-pixel accuracy is calculated through a two-dimensional corrected distance method, and the final detection result is output.

2. The astronomical image star detection and sub-pixel centering method based on FPGA according to claim 1, characterized in that: In step S1, the multi-pixel parallel input is converted into a single-pixel serial output, and the row and column numbers of the pixels are output synchronously, and the serially transmitted pixels are binarized: Among them, v (m,n) Represents the grayscale value of the input image m rows and n columns, T (p,q) represents the value of the pth row and qth column of the threshold array, u (m,n) Represents the gray value of the mth row and nth column of the binarized image, and p,q and m,n satisfy the following conditions: in, It means rounding down. M and N are divisible by P and Q respectively. When P=1 and Q=1, it is equivalent to using the global threshold.

3. The astronomical image star detection and sub-pixel centering method based on FPGA according to claim 1, characterized in that: In step S2, each pixel is scanned row by row from top to bottom, neighborhood detection is performed to generate star labels, and corresponding star parameters are recorded.

4. The astronomical image star detection and sub-pixel centering method based on FPGA according to claim 3, characterized in that: Neighborhood detection is specifically as follows: Update the pixel star image label record of the previous row. If the current row to be processed is the first row, initialize it to unassigned label; Read the pixels in sequence, and when the value of the pixel is not zero, read the star labels of the pixels to the upper right, above, upper left, and left of the pixel, and check in a counterclockwise direction whether the pixels to the upper right, above, upper left, and left of the pixel are allocated. Based on the detection results, perform the following: When the constellation labels of adjacent pixels have not been assigned, a new constellation label is assigned to the pixel; When the upper right pixel is not assigned, the upper pixel, the upper left pixel and the left pixel are detected in sequence, and when any pixel is assigned, the pixel is newly assigned the star sign corresponding to the assigned pixel; When allocating the upper right pixel, the pixel is newly allocated the same constellation label as the upper right pixel.

5. The astronomical image star detection and sub-pixel centering method based on FPGA according to claim 4, characterized in that: Unassigned pixels are set to 0.

6. The astronomical image star detection and sub-pixel centering method based on FPGA according to claim 4, characterized in that: In the case of upper right pixel allocation, the following is performed for label conflicts: When allocating the upper left pixel, determine whether the upper left pixel star sign is equal to the upper right pixel star sign, and if they are not equal, merge the upper left pixel star sign into the upper right pixel star sign; When allocating the left pixel, determine whether the left pixel star sign is equal to the upper right pixel star sign, and if they are not equal, merge the left pixel star sign into the upper right pixel star sign.

7. The astronomical image star detection and sub-pixel centering method based on FPGA according to claim 1, characterized in that: In step S3, the star parameter records are scanned, and when the coordinate difference between the row being detected and the row where the lowest pixel contained in the star is located is greater than 1, the star is completely detected and the recorded star parameters are output.

8. The astronomical image star detection and sub-pixel centering method based on FPGA according to claim 1, characterized in that: In step S4, the centroid of the star image with sub-pixel accuracy is calculated as: Among them, u (m,n) is the binary grayscale value at (m,n), x m ,y n is the pixel coordinate value at (m,n), and the value range of m and n is the upper, lower, left and right boundary limit area of ​​the star image; ∑ m ∑ n (u (m,n) ·x m ),∑ m ∑ n (u (m,n) ·y n ),∑ m ∑ n u (m,n) They are respectively the cumulative sum x0 of the product of the pixel grayscale value and its column coordinates, the cumulative sum y0 of the product of the pixel grayscale value and its row coordinates, and the total grayscale value L.

9. An astronomical image star detection and sub-pixel centering system based on FPGA, used to implement the astronomical image star detection and sub-pixel centering method based on FPGA according to any one of claims 1 to 8, characterized in that: The system comprises: Image input module: receives parallel input of multiple pixels and converts them into single pixel serial output; Parameter storage module: stores pixel segmentation threshold array, array size, image background noise, minimum number of pixels of output star image and minimum signal-to-noise ratio; threshold array includes global threshold or block threshold; Binarization segmentation module: reads the array size stored in the parameter storage module, reads the corresponding threshold value according to the row and column where the input pixel is located, and binarizes the pixel; Star detection module: scans each pixel, detects its neighborhood, and sends star parameter update, merging signals and data to the star parameter cache management module; Astrological parameter cache management module: manages astrological parameters and outputs the detected astrological images; Sub-pixel centering calculation module: uses the two-dimensional corrected distance method to calculate the centroid position of the star image; Detection result output module: reads the configuration parameters of the parameter storage module and outputs the star detection results.

10. The astronomical image star detection and sub-pixel centering system based on FPGA according to claim 9, characterized in that: The detection result output module outputs the minimum number of pixels, or calculates the signal-to-noise ratio and outputs the minimum signal-to-noise ratio: Among them, L is the total gray value, n is the number of pixels in the star image, and σ is the image background noise.

Images