Large array CMOS detector image windowing translation and output method

Abstract

The present application relates to a kind of big array CMOS detector image windowing translation and output method, first determine the detector data channel needing to split;The image data of split detector data channel is stored in RAM backup;The pixel position in the split detector data channel corresponding to the start and end pixel of each digital transmission channel is mapped into the data integration logic module of FPGA with the parameter of window translation adjustment value, data integration logic module reads out image data from RAM according to the set start and end position of pixel, completes image data integration output.Sending window translation adjustment value, without changing the structure of FPGA software, quickly and flexibly realize output window translation processing.The method of the present application is flexibly switched by parameter injection, realizes flexible window setting when big array CMOS detector windowing imaging, shortens development cycle, simplifies test process, and increases the flexibility of camera application.

Description

Technical Field

[0001] This invention relates to the field of detector imaging technology, and in particular to a method for windowing, shifting, and outputting images from a large-area CMOS detector. Background Technology

[0002] In aerospace remote sensing applications, large-area CMOS detectors often have pixel sizes in the hundreds of megabytes. In practice, it is necessary to perform windowing processing on large-area CMOS detectors to capture the effective field of view and reduce the number of output pixels.

[0003] The image data of the large-area CMOS detector is output in parallel through multiple data channels and connected to the FPGA, which then receives and transfers the parallel image data of the detector.

[0004] Because large-area CMOS detectors use multiple data channels and high-speed output of image data, after the multi-channel image data from the detector is input to the FPGA for data integration, it needs to be output simultaneously and in parallel through multiple data transmission channels. When the number of data channels output by the detector and the number of data transmission channels of the camera cannot be evenly matched, the actual number of pixels output by each data transmission channel cannot correspond to the number of data channels of the detector as an integer.

[0005] To achieve standardized design of camera data transmission channel data format, each data transmission channel is required to transmit image data of a fixed specification, with the same amount of image data transmitted per channel. In existing methods, when dividing the detector data channels, each data transmission channel corresponds to an integer number of detector data channels, outputting the number of pixels corresponding to that integer number of detector data channels. This results in different numbers of data channels and pixels transmitted per data transmission channel. Image acquisition equipment and data processing software require specific design based on the number of detector data channels output by each data transmission channel. Furthermore, when adjusting the size and position of the detector output window, the amount of image data output by each data transmission channel changes, making data transmission protocol modifications complex and cumbersome.

[0006] This type of detector has poor flexibility in its metadata partitioning structure. When the window position needs to be adjusted, the FPGA software needs to be updated. Since the detector window position needs to be confirmed after the entire system is assembled and adjusted, any program changes or software updates made at this time have a wide impact and take a long time. Such problems must be avoided in aerospace engineering practice. Changing the detector's output window position will alter the image data transmission protocol, significantly impacting the interface data sheet and downstream acquisition equipment. Therefore, during the overall assembly and adjustment, the window position accuracy needs to be controlled by adjusting the mechanism, which further increases the design and assembly complexity. Summary of the Invention

[0007] To address the problems of poor flexibility in the data partitioning structure of detector images, large impact range of program changes and software annotations, long modification time, and high design and adjustment difficulty in existing methods, this invention proposes a method for image windowing translation and output of large-area CMOS detectors.

[0008] To solve the above problems, the present invention adopts the following technical solution:

[0009] A method for windowing, shifting, and outputting images from a large-area CMOS detector, under multi-channel data transmission, includes the following steps:

[0010] Step 1: Calculate the number of pixels M transmitted by each data transmission channel based on the ratio of the number of camera data transmission channels N to the number of pixels Q per row of images. Then, determine the detector data channels that need to be split based on the ratio of the number of pixels M transmitted by each data transmission channel to the amount of data A per channel of the detector and the pixel output range.

[0011] Step 2: Back up and store the image data from the split detector data channels in RAM;

[0012] Step 3: The pixel positions and window translation adjustment values ​​in the split detector data channels corresponding to the start and end pixels of each data transmission channel are transmitted to the data integration logic module of the FPGA in the form of parameter mapping. The data integration logic module reads the image data from RAM according to the set start and end pixel positions and completes the image data integration output.

[0013] When adjusting the window position, the effective range of the window translation adjustment value is calculated based on the image data splitting position of the detector channel. The window translation adjustment value parameter setting within the effective range does not change the data storage structure inside the FPGA. If the window translation adjustment value exceeds the effective range, it is necessary to change the detector data channel number corresponding to the data transmission channel, re-back up the data, and then switch the output RAM channel number to realize the selection control of the output window image data.

[0014] The beneficial effects of the present invention are as follows: (1) The present invention provides a data transmission output structure based on the equal distribution and real-time adjustment of the number of output pixels of the detector, and gives the internal image data storage structure and windowing translation method of the FPGA. The method of the present invention is not limited by the number of data channels of the detector, and the windowing position can be arbitrarily adjusted within a certain pixel space. In the experiments of windowing imaging, window movement, and camera focal plane calibration of large-area CMOS detector, the window can be flexibly switched by the method of parameter annotation, realizing more flexible window settings when the large-area CMOS detector is windowed. This parameter-based large-area CMOS detector window translation and output method can be applied to both linear CMOS detectors and TDI-CCD detectors. (2) The consistent design of the multi-channel data transmission protocol reduces the development difficulty of fast-view acquisition equipment and image interpretation software, shortens the development cycle, simplifies the testing process, provides convenience for subsequent image data preprocessing, and increases the flexibility of camera application. Attached Figure Description

[0015] Figure 1 This is a schematic diagram of the RAM storage structure of the detector channel data in the FPGA in an embodiment of the present invention;

[0016] Figure 2 A schematic diagram of the image data processing structure and window translation settings for a large-area CMOS detector;

[0017] Figure 3 A schematic diagram of window translation settings and image output for a TDI-CCD detector. Detailed Implementation

[0018] The technical solution of the present invention will now be described in detail with reference to the accompanying drawings and preferred embodiments.

[0019] This invention provides a method for windowing and shifting and outputting images from a large-area CMOS detector. Under multiple data transmission channels, the method includes the following steps one to three.

[0020] Step 1: Calculate the number of pixels M transmitted by each data transmission channel based on the ratio of the number of camera data transmission channels N to the number of pixels Q per row of image. Then, determine the detector data channels that need to be split based on the ratio of the number of pixels M transmitted by each data transmission channel to the amount of data A per channel of the detector and the pixel output range.

[0021] This invention provides a multi-channel image data storage structure for a large-area CMOS detector, used to quickly implement a detector output pixel splitting strategy and achieve unified data format across data transmission channels. This structure calculates the detector channel number to be split based on the output data volume per detector channel and the output data volume per data transmission channel, and backs up the pixel data for that channel, meaning two identical sets of stored data each correspond to one data transmission channel.

[0022] Specifically, when designing the data storage structure of a large-area CMOS detector, the number of pixels M transmitted per data transmission channel is first calculated based on the number N of camera data transmission channels. Let Q be the number of pixels per row of images, such that N×M=Q (if this cannot be satisfied, zeros can be added appropriately) to ensure consistent image data format across all data transmission channels. Let A be the data volume per channel of the detector; then M / A represents the number of channels in the CMOS detector corresponding to each data transmission channel. Based on the calculation result of M / A and the pixel output range, the detector data channel numbers that need to be split are determined, thus identifying the detector data channels that need to be split.

[0023] After this splitting process, the number of detector channels corresponding to each data transmission channel is different, possibly [M / A], [M / A]+1, or [M / A]+2, where [] indicates rounding.

[0024] Step Two: After determining the detector data channels that need to be split, back up and store the image data of those channels in RAM, such as... Figure 1 As shown, the image data of detector channels 2, 6, and 10 are simultaneously stored in two RAMs.

[0025] Step 3: By sending command parameters, the pixel positions A1, A2, A3, A4, and window shift adjustment values ​​T1, T2 in the split detector data channels corresponding to the start and end pixels of each data transmission channel are transmitted to the FPGA data integration logic module in a parameter mapping manner. Finally, the data integration logic module reads the image data from RAM according to the set pixel start and end positions, completing the image data integration and output. When reading data, the data integration logic module reads a portion of the data from each of the two backup RAMs for each split detector data channel, and the two portions are combined to form the complete data.

[0026] This embodiment provides a method for adjusting the window position of a large-area CMOS detector while ensuring that the data format of the data transmission channel remains unchanged. By sending window translation parameters, output window translation processing can be quickly and flexibly implemented without altering the FPGA software structure. The configurable window translation parameters (i.e., window translation adjustment values) corresponding to each data transmission output structure are set with valid ranges. Parameter settings within the valid range do not change the internal data storage structure of the FPGA; parameter settings outside the valid range will change the internal data storage structure of the FPGA.

[0027] After determining the window position and data transmission output structure of the large-area CMOS detector, a window position translation adjustment function is added to adjust the window position in real time during product debugging.

[0028] Using the initial pixel position 'a' of the detector corresponding to data transmission channel 1 as a reference, bidirectional translation is performed, with the window translation adjustment value set to T. When 'a' is the starting pixel of a certain channel of the detector, T ≥ 0 without considering changes to the internal storage structure of the FPGA. When the detector pixel data requires continuous output, the window translation adjustment value T can simultaneously realize the adjustment function of the start and end position parameters of all data transmission channels.

[0029] The effective range of the window shift adjustment value T is calculated based on the image data splitting position of the detector channel. Once the data transmission structure and the detector channel data's internal data storage structure within the FPGA are confirmed, the effective range of T can be determined. When adding parameters, ensure that the window shift adjustment value T is within a reasonable range.

[0030] If the window shift adjustment value T exceeds the effective range of a single channel adjustment, the detector data channel number corresponding to the data transmission channel needs to be changed. In this case, the corresponding data channel of the detector needs to be backed up again according to the adjustment range, and then the output RAM channel number is switched to achieve selection control of the output window image data.

[0031] The setting range of the window shift adjustment value T can be determined according to the distribution value of the number of pixels of the detector split data channel in the two data transmission channels. That is, after the detector data channel is split into two parts, the number of pixels in each part (denoted as Nd) is greater than 0 and less than A, i.e., 0 < Nd < A, where A is the amount of data per channel of the detector.

[0032] The method of the present invention is not limited by the number of detector data channels and can arbitrarily adjust the window position within a certain pixel space. In experiments such as window imaging, window movement, and camera focal plane calibration of large-area CMOS detectors, the window can be flexibly switched by adding parameters, thus realizing more flexible window settings when performing window imaging of large-area CMOS detectors.

[0033] For single-channel output structures, another embodiment of the present invention provides a windowing and image acquisition method for single-channel and multi-channel image data sources of detectors. This method is applicable to the image data windowing design and window translation design of linear CMOS detectors, TDI-CCD detectors, etc.

[0034] This method enables flexible control over the detector's output image, increasing design flexibility and setup methods in subsequent debugging, testing, and camera focal plane determination experiments. The consistent design of the multi-channel data transmission protocol reduces the development difficulty of fast-view acquisition equipment and image interpretation software, shortens the development cycle, and simplifies the testing process.

[0035] In a single data transmission channel, the image windowing translation and output method includes the following steps:

[0036] Step 1: Integrate the multi-channel image data of the large-area CMOS detector;

[0037] Step 2: Perform windowing processing on the integrated image data to complete the windowed output of the image data; during window translation processing, perform dynamic windowing based on the starting pixel position and the number of pixels in the window.

[0038] When performing windowing on multi-channel image data of a large-area CMOS detector under a single data transmission channel, it is necessary to first complete the integration of multi-channel image data, and then perform windowing processing on the integrated image data. When implementing window translation processing, it is only necessary to provide the starting pixel position and the number of window pixels to dynamically open the window. This windowing method simplifies the design and does not reduce the output line frequency of the detector image data.

[0039] In a single data transmission channel, when performing windowing processing on single-channel image data of a large-area CMOS detector, windowing is performed during image data sampling to avoid storing redundant image data in the FPGA, thus saving internal storage space in the FPGA.

[0040] Figure 2 This is a schematic diagram illustrating the image data processing structure and window translation settings for a large-area CMOS detector. Figure 2As shown, taking a large-area CMOS detector imaging example, image data is output in parallel from four TLK2711 data transmission channels. Each data transmission channel outputs 2048 pixels, for a total of 8192 pixels across the four channels. Each detector channel outputs 504 pixels, and the 8192×8192 pixel output is taken from the center of the detector, corresponding to channels 2 through 18. Mapping is performed based on the output pixel positions: data transmission channel 1 outputs data from detector channels 2 through 6; data transmission channel 2 outputs data from detector channels 6 through 10; data transmission channel 3 outputs data from detector channels 10 through 14; and data transmission channel 4 outputs data from detector channels 14 through 18. Therefore, the image data from channels 6, 10, and 14 are divided into two parts, output from different data transmission channels, and this data is backed up and stored internally within the FPGA.

[0041] Each detector data channel outputs 504 pixels. The positions of the split data channels in the output window are as follows: Figure 2 As shown, based on 0 < Nd < 504 and 127 + A1 < 504, we know that A1 < 377. Therefore, under the design of transmitting 5 detector data channels per data transmission channel, the adjustable pixel range is calculated to be [0 ~ 376], which means the maximum number of adjustable pixels is 376. When adjusting within this range, there is no need to adjust the RAM storage structure.

[0042] like Figure 3 As shown, taking the imaging process after replacing a large-area CMOS detector with a TDI-CCD detector as an example, the P-band image data of the TDI-CCD detector is output to the FPGA from two channels in opposite directions. Specifically, the first channel (RAM1) outputs 1 pixel first, and the second channel (RAM2) outputs 1536 pixels first, with 768 pixels per channel, totaling 1536 pixels. The P-band data transmission outputs 1440 pixels of image data per channel with a window, and the window position can be set. From a simplified design perspective, the image data of the second channel (RAM2) in the P-band image data is reversed and then concatenated with the image data of the first channel (RAM1). Then, window selection is performed. By setting parameter A1, the image data output position is selected after image data integration, completing the windowed output of the image data. When outputting P-band data from multiple channels, the method is similar if the window data needs to be extracted from multiple channels.

[0043] The B-band image data from the TDI-CCD detector is output to the FPGA via a single channel. Based on the detector drive timing design, the B-band image data is divided into two parts. During image data cropping, only the windowed image data is buffered; therefore, windowing is performed synchronously during data sampling. There are 768 multispectral image data points. By setting the A2 value and adjusting the detector drive timing, the valid sampling time for each image data segment is selected, and 720 pixels are buffered and output.

[0044] The above method enables dynamic adjustment of the window position in the windowed mode of detector image data, which facilitates subsequent image data preprocessing and increases the flexibility of camera applications.

[0045] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0046] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention patent should be determined by the appended claims.

Claims

1. A method for windowing, shifting, and outputting images from a large-area CMOS detector, characterized in that, Includes the following steps: Step 1: Under multiple data transmission channels, calculate the number of pixels M transmitted by each data transmission channel based on the ratio of the number of camera data transmission channels N to the number of pixels Q per row of images. Then, determine the detector data channels that need to be split based on the ratio of the number of pixels M transmitted by each data transmission channel to the amount of data A per channel of the detector and the pixel output range. Step 2: Back up and store the image data from the split detector data channels in RAM; Step 3: The pixel positions and window translation adjustment values ​​in the split detector data channels corresponding to the start and end pixels of each data transmission channel are transmitted to the data integration logic module of the FPGA in the form of parameter mapping. The data integration logic module reads the image data from RAM according to the set start and end pixel positions and completes the image data integration output. When adjusting the window position, the effective range of the window translation adjustment value is calculated based on the image data splitting position of the detector channel. The window translation adjustment value parameter setting within the effective range does not change the data storage structure inside the FPGA. If the window translation adjustment value exceeds the effective range, it is necessary to change the detector data channel number corresponding to the data transmission channel, perform data backup again, and then switch the output RAM channel number to achieve selection control of the output window image data.

2. The method for image windowing translation and output of a large-area CMOS detector according to claim 1, characterized in that, The setting range of the window translation adjustment value satisfies: 0 < Nd < A, where Nd is the number of pixels in the two data transmission channels of the split detector data channel.

3. The method for image windowing translation and output of a large-area CMOS detector according to claim 1, characterized in that, In a single data transmission channel, the image windowing translation and output method includes the following steps: Step 1: Integrate the multi-channel image data of the large-area CMOS detector; Step 2: Perform windowing processing on the integrated image data to complete the windowed output of the image data; When performing window panning, the window is dynamically opened based on the starting pixel position and the number of window pixels.

4. The method for image windowing translation and output of a large-area CMOS detector according to claim 3, characterized in that, The large-area CMOS detector was replaced with a linear CMOS detector.

5. The method for image windowing translation and output of a large-area CMOS detector according to claim 3, characterized in that, The large-area CMOS detector was replaced with a TDI-CCD detector.

6. The method for windowing, shifting, and outputting images of a large-area CMOS detector according to claim 5, characterized in that, For P-band image data, during the integration operation in step one, the second channel image data in the 2-channel output image data is reversed and then connected end to end with the first channel image data for integration.

7. The method for image windowing translation and output of a large-area CMOS detector according to claim 5, characterized in that, For B-band image data, a window is directly captured during the image data sampling process, and only the window image data is cached.

Images