SDI transmission method of remote sensing camera push-broom imaging data based on FPGA

Abstract

The invention discloses an SDI (Serial Digital Interface) transmission method for remote sensing camera push-broom imaging data based on an FPGA (Field Programmable Gate Array), relates to the technical field of remote sensing camera imaging data transmission, and solves the problems of high cost, large interface size, heavy cable and the like in the existing imaging data transmission. Obtaining a pixel rate matched with the SDI transmission; the data are stored in the FIFO according to the matching pixel rate transmitted by the SDI; sequentially reading data in the FIFO, filling invalid data generated in a rate mismatching area in a transmission process to obtain an SDI data stream of the push-broom camera, and sending the SDI data stream to an external receiving end through a high-speed transceiver; and an external receiving end receives data through the high-speed transceiver to obtain effective data. According to the method, protocol control parameters are adjustable, and remote sensing push-broom camera data streams with different rates can be met through an SDI interface with a fixed rate.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing camera imaging data transmission, and specifically relates to an SDI transmission method for push-broom imaging data of a remote sensing camera based on FPGA. Background Art

[0002] Currently, common image interfaces in the aerospace field are mostly Cameralink, TLK coaxial interfaces, and CoaXPress. The Cameralink protocol is simple, has a high transmission speed, and is easy to expand, but it is large in size and heavy in cable, which is inconvenient in structural design and transportation; the TLK coaxial interface has a simple structure and a small interface, but the transmission speed is slow; the CoaXPress interface is also a coaxial interface, which has the characteristics of long transmission distance, scalability, and high speed, but the protocol is complex and the cost is high.

[0003] The SDI interface is also a coaxial cable interface, which has the characteristics of high speed, simple and mature protocol, light structure, and high transmission speed, and is widely used in the field of high-speed image transmission. Therefore, it is necessary to develop a transmission method for remote sensing push-broom imaging data for SDI transmission in terms of reducing design complexity, reducing weight, and increasing transmission speed.

[0004] Therefore, the present invention provides an SDI transmission method for push-broom imaging data of a remote sensing camera based on FPGA. Summary of the Invention

[0005] In order to solve the problems of high cost, large interface size, and heavy cable in the transmission of existing imaging data, the present invention provides an SDI transmission method for push-broom imaging data of a remote sensing camera based on FPGA.

[0006] The SDI transmission method for push-broom imaging data of a remote sensing camera based on FPGA is implemented by the following steps:

[0007] Step 1: The FPGA decodes the received push-broom imaging data of the remote sensing camera to obtain data that meets the SDI transmission bandwidth.

[0008] Step 2: The FPGA calculates the pixel rate of the received push-broom imaging data of the remote sensing camera to obtain a matching pixel rate for SDI transmission.

[0009] Step 3: Store the data obtained in Step 1 that meets the SDI transmission bandwidth into the first FIFO buffer according to the matching pixel rate for SDI transmission in Step 2.

[0010] Step 4: The FPGA sequentially reads the data in the FIFO, fills the invalid data generated in the rate mismatch area during the transmission according to the matching pixel rate transmitted by the SDI, obtains the SDI data stream of the push-broom camera, and sends it to the external receiving end through the high-speed transceiver of the FPGA;

[0011] Step 5: The external receiving end receives the data through the high-speed transceiver of the FPGA, and identifies according to the line number, frame valid, and line valid flags in the SDI data stream to obtain the valid data;

[0012] Step 6: The external receiving end transmits the valid data back to the FPGA for decoding and stores it in the second FIFO buffer, and reads it at the original push-broom camera rate to restore the original remote sensing camera push-broom imaging data.

[0013] Advantages of the present invention:

[0014] Based on the SDI transmission protocol, this method designs an SDI-based remote sensing camera data stream transmission protocol that can be deployed on the FPGA by performing calculations such as clock matching on the SDI protocol and the remote sensing camera data protocol. The remote sensing camera data stream is sorted and sent at the output end, and the data stream can be decoded into the original camera data stream at the receiving end, thereby using the SDI interface to replace the cameralink interface.

[0015] The protocol control parameters of the method of the present invention are adjustable, and the data streams of remote sensing push-broom cameras with different rates can be satisfied through the SDI interface with a fixed rate. Description of the Drawings

[0016] Figure 1 It is a schematic diagram of the SDI transmission method of the remote sensing camera push-broom imaging data based on the FPGA of the present invention;

[0017] Figure 2 It is a matching data diagram of the remote sensing camera output data stream and the SDI data stream;

[0018] Figure 3 It is a schematic diagram of the SDI single-frame video structure;

[0019] Figure 4 It is a schematic diagram of the parallel data transmission format of the SDI transmission interface;

[0020] Figure 5 It is a schematic diagram of the SDI protocol data stream;

[0021] Figure 6 It is an FPGA simulation effect diagram;

[0022] Figure 7 It is a transmission schematic diagram of the remote sensing camera based on the SDI interface. Detailed implementation mode

[0023] Combined with Figures 1 to 7 This implementation mode is described. For the SDI transmission method of the push-broom imaging data of a remote sensing camera based on FPGA, since most of the current remote sensing camera driving and data transmission are based on FPGA, a protocol development that can solve the requirements of miniaturization of the transmission interface and lightweight of the cable is carried out on the basis of FPGA. The specific implementation steps of this method are as follows:

[0024] Step 1: Decode the remote sensing push-broom imaging data, arrange the data, and merge it into data that conforms to the SDI transmission bandwidth;

[0025] Step 2: Then rearrange it according to the matching pixel speed of SDI transmission and store it in the first-in first-out storage queue;

[0026] Step 3: Calculate based on the pixel rate of the push-broom imaging data to obtain the matching pixel rate for SDI transmission;

[0027] Step 4: The FPGA sequentially reads the data in the first FIFO buffer, fills the invalid data stream part generated in the rate mismatch area during the transmission process based on the calculated matching pixel rate, obtains the SDI data stream of the push-broom camera, and sends it to the external receiving end through the high-speed transceiver of the FPGA;

[0028] Step 5: The external receiving end receives the data through the high-speed transceiver of the FPGA, identifies based on the flags such as line number, frame valid, and line valid in the SDI data stream, and obtains the valid data;

[0029] Step 6: Finally, decode the valid data, store it in the second FIFO buffer, read it at the original push-broom camera rate, restore it to the original data, and use it for subsequent processing.

[0030] Such as Figure 1 shown Figure 1 is the schematic diagram of the SDI transmission method for the push-broom imaging data of a remote sensing camera based on FPGA. In the figure, the inside of the FPGA is split into multiple modules according to the transceiver flow direction and functional nodes of the data stream. Specifically, it includes a data processing module, a 20-bit recombination module, a first FIFO buffer, a frame construction encoding module, a frame construction decoding module, a second FIFO buffer, and a 20-bit de-recombination module 、High-speed transceiver and SDI protocol module; when the remote sensing camera transmits the push-broom imaging data externally, the remote sensing camera generates an imaging data stream (remote sensing camera data stream) under the drive of the FPGA and flows into the FPGA. At this time, the image data does not conform to the arrangement layout of the real spatial position. After the image signal is converted into the real spatial layout by the data processing module and then passes through the 20-bit recombination module, it is sorted into 20-bit-wide data suitable for the SDI transmission bandwidth, and then enters the first FIFO buffer for rate matching and data length conversion waiting, and is transmitted to the frame construction and encoding module that conforms to the SDI protocol. The frame construction and encoding module splices and fills the imaging data stream to construct two-dimensional frame data that conforms to the SDI transmission protocol, and then sends the two-dimensional frame data to the high-speed transceiver through the SDI protocol module according to the SDI transmission protocol. The serial interface of the high-speed transceiver transmits the serial signal to the external receiving end through the coaxial cable via the BNC physical interface and the SDI driver.

[0031] When the FPGA receives the remote sensing camera imaging data that conforms to the SDI protocol transmitted by the external receiving end, the data stream is transmitted to the FPGA through the SDI equalizer and the BNC interface, and the serial data stream that conforms to the SDI protocol is obtained by decoding through the high-speed serial interface. The stream data is converted into two-dimensional frame data through the SDI protocol module. The two-dimensional frame data extracts the valid data part of the remote sensing push-broom camera through the frame construction and decoding module, and is cached through the second FIFO buffer. Then, it is parsed into image data that conforms to the size of the remote sensing camera protocol through the 20-bit de-recombination module, and the image data of the remote sensing camera is parsed at its original data rate to obtain an imaging data stream that conforms to the original camera rate and bandwidth.

[0032] In this embodiment, the format of the remote sensing camera push-broom imaging data stream is usually three parts of parallel data, namely frame valid (FVAL, 1 bit), line valid (LVAL, 1 bit), and pixel data (DATA, 10 bit - 12 bit). The data stream continuously outputs within a cycle according to the instruction.

[0033] In this embodiment, the SDI transmission is at a constant speed, and its transmission speeds have four modes: 1.5 Gb / s, 3 Gb / s, 6 Gb / s, and 12 Gb / s. The first three modes can all be completed by the serial interface design of a high-speed transceiver of an FPGA. The SDI at a rate of 12 Gb / s can be completed by the interface design of a mid- to high-end FPGA high-speed transceiver. The speeds of push-broom cameras are different. The single-line output data N is the number of window openings n multiplied by the width W. The output data rate is often obtained by multiplying the ground movement speed of 7062 m / s by N and then dividing by the ground resolution GSD. Taking the camearlink rate as a reference, its maximum data rate is 6.4 Gb / s, and the upper limits of the common data rates are 2 Gb / s (the maximum rate in the base mode), 4 Gb / s (the maximum rate in the medium mode), and 5.3 Gb / s (the maximum rate in the full mode). Taking the SDI transmission protocol of 2048×1080@60fps that complies with SMPTE ST 2048 as an example, the effective data part of its transmission is 2048×1080×60 = 132,710,400 pixels / s, which exceeds the base mode of camearlink. Taking a ground resolution of 1 m as an example, it can support the single-line camera output of 17,617 pixels.

[0034] In this embodiment, taking 4096×2160@30fps that complies with SMPTE ST 2048-1 as an example, the effective data part of its transmission is 4096×2160×30 = 265,420,800 pixels / s. Taking a ground resolution of 1 m as an example, it can support the single-line camera output of 35,235 pixels. That is, a single SDI interface can meet the output requirements of a 5-window remote sensing push-broom camera with a width of 7000 m, while the same data stream requires two camearlink interfaces to meet. The speed upper limit of a single cameralink interface is 2.0 Gb / s. Calculated based on 10 bits per pixel, the pixel transmission rate of a single cameralink interface is: 200,000,000 pixels / s, which is less than 265,420,800 pixels / s, so two interfaces are required.

[0035] The SDI transmission method described in this embodiment constructs a data frame that complies with the SDI protocol manually, while also complying with the video protocol regulations of ITU. Manually actively locates the data interval between EVA (End of active video, effective video end signal) and SAV (Start of active video, effective video start signal), calculates the conversion relationship based on the speed of the camearlink data stream and the speed of the SDI protocol, fills the data into the designed SDI transmission frame, and then conducts the transmission. For example Figure 2As shown in the figure, this is a schematic diagram of the CMOS data stream of a remote sensing camera being constructed into an SDI data stream. The SDI protocol with a fast transmission speed is selected to transmit the slow CMOS video stream. Among them, the SDI data stream has a fast speed and the CMOS data stream has a slow speed. However, not all of the SDI serial data stream is valid data. There is auxiliary data for valid video recognition in the auxiliary protocol. The red part of the valid data needs to fill and supplement the auxiliary data by itself when being converted into the SDI data stream.

[0036] Taking the above SDI transmission protocol of 2048×1080@60fp as an example, the upper limit of its transmission rate is 132,710,400 pixels / s. To make the push-broom data match the SDI data, it is necessary to calculate the push-broom data during the SDI idle period, as well as the difference between the push-broom data rate and the SDI rate. The time of its idle period is the number of line intervals LN specified in the video protocol that conforms to CEA861 multiplied by the line period T, and then multiplied by the push-broom output data rate V, to obtain the total number of pixels MM of the push-broom camera data that needs to be stored during the idle period. For this reason, a FIFO needs to be specially designed. The size of the FIFO is MM, and its width is WW;

[0037] MM = LN×T×V

[0038] WW = FVAL + LVAL + DATA

[0039] At the same time, in order to distinguish whether the filled data in the SDI protocol belongs to the valid data of remote sensing push-broom or the filled invalid data, when constructing the SDI video protocol, for its 20-bit single-cycle bit width, the high two bits are designed as XX. At this time, there are still 18 bits left, which can be designed as supplementary bits + FVAL + LVAL + DATA. The sum of the number of supplementary bits and the number of DATA bits is 16, which can meet the transmission requirements of 8-bit to 16-bit cameras.

[0040] For the high XX bits, when transmitting valid data, it is 1, and when there is no valid data, it is 00. The 20-bit of the invalid data is 3FFFFh.

[0041] As Figure 3 shown, in this embodiment, Figure 3 This is the frame timing relationship in the SDI transmission system. L1-L6 are respectively the first line of a frame, the last line of the digital frame blanking, the first line of the valid image, the last line of the valid image, the first line of the digital frame blanking, and the last line of a frame. Figure 4It is the parallel data transmission format for the SDI transmission interface. The input data stream of SDI complies with the CEA-861 protocol, which includes two parts: valid video and blanking data, and its data bandwidth per cycle is 20 bits. In SDI data, the blanking and valid regions are distinguished by the SAV and EVA data therein to determine the valid and frame valid signals. The blanking region is assisted in data determination by designing four start and end 20-bit data, which are 3FFh, 000h, 000h, and XYZh respectively. Each line of valid data starts with SAV as the start signal and ends with EVA as the end signal. At the end of EVA, there are line numbers and CRC check codes to assist in data verification and statistics, and then there is the blanking auxiliary data area 40200h. The available part of the push-broom imaging data of the remote sensing camera during the overall transmission process is the valid data part of SAV and EVA, and its blanking part can also be filled in subsequent development.

[0042] As Figure 5 shown, in this embodiment, when the remote sensing camera generates valid data, the specific method of SDI transmission is as follows:

[0043] The valid data stream of the remote sensing camera is sorted out by the FPGA to obtain the corresponding LVAL, FVAL, and DATA.

[0044] The entire SDI transmission process is in a pipeline form. The data stream is sent in a serial form and frame format, and the data frame corresponding to the line data has been constructed before sending. The upper part of the remote sensing image data of the red dotted line, after reconstruction and speed matching, is transmitted through the nth frame of SDI, and then the subsequent remote sensing image data is transmitted by constructing the (n + 1)th frame of the SDI transmission construction frame. During the SDI transmission of auxiliary data, the valid data of the remote sensing camera needs to be cached and reorganized into 20 bits. During the SDI transmission of valid data, the data is sent out from the first FIFO buffer and constructed into an SDI transmission reconstruction frame for external transmission, completing the transmission of the remote sensing camera image data stream based on SDI.

[0045] In this embodiment, according to the calculated value of clock matching, the FIFO depth is designed; the width of the FIFO is equal to the width of FVAL + LVAL + DATA. This FIFO is designed as an asynchronous clock FIFO. The input part includes a write clock, a write enable, and write valid data bits. Its valid data is the valid data of remote sensing push-broom, which are FVAL + LVAL + DATA respectively. Among them, the write clock matches the push-broom data clock, and the write enable is always at a high level after the remote sensing camera starts working and is at a low level after the remote sensing camera stops working.

[0046] The output terminals of the asynchronous FIFO are the read clock, read enable, read valid data bits, FULL flag bit, and EMPTY flag bit; the read clock matches the SDI data clock and is a different fixed value in different SDI modes; the read enable is equal to the inversion of EMPTY and ANDed with the valid flag of the SDI construction module; when the read enable is high, the read valid data is FVAL+LVAL+DATA, and when the read enable is low, it is all high level.

[0047] In this embodiment, after the remote sensing camera starts working and outputs valid data, the frame encoding module starts working and outputs the SDI data stream. Specifically: the inputs of this module are the SDI data clock, the frame construction module enable, and the output data of the asynchronous FIFO, the total number of rows R of the constructed frame, the total number of columns L of the frame, the column start L0, the column width L1, the row start R0, and the row width R1. The outputs of this module are the module data valid flag and the SDI video stream. When the frame construction module enable is turned on, the internal column and row counts of the frame construction module start. Taking the total number of rows and columns of the frame as the maximum count value, both starting from 0 as the initial count value. When the total number of columns is equal to the maximum value, the number of rows is incremented by one and the number of columns is reset to zero. When the column number Yn is greater than L0 and less than L1+L0, and the row number is greater than R0 and less than R0+R1, the module data valid flag is high level, corresponding to the valid video part in the SDI video transmission frame. At this time, if the FIFO read enable is high, the data in the asynchronous FIFO is read and constructed into 20-bit format valid data of the remote sensing camera and output to the SDI video stream. If the read enable of the FIFO is low at this time, the data in the SDI video stream is 3FFFFh. At other times, the module data valid flag is low level, and the data in the SDI video stream is auxiliary data. The SDI video stream is directly connected to the SDI transmission protocol module of the FPGA and sent to the SDI driver through the high-speed transceiver of the FPGA to generate the selected SDI format data signal. Its FPGA simulation diagram is as Figure 6 shown.

[0048] As Figure 7 shown, the remote sensing camera obtains remote sensing image data in units of rows through pushbroom imaging. The pixels of the remote sensing image data are rearranged into an SDI construction frame according to the SDI transmission method of the remote sensing camera pushbroom imaging data based on the FPGA described in this embodiment, and the data of the SDI construction frame is transmitted through the SDI protocol and finally decoded into the actual image of the remote sensing camera to complete the transmission.

[0049] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0050] The above-described embodiments merely represent several implementation manners of the present invention. The description thereof is relatively specific and detailed, but it should not be construed as a limitation on the scope of the invention patent. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present invention, several modifications and improvements can still be made, and these all fall within the protection scope of the present invention. Therefore, the protection scope of the present invention patent shall be subject to the appended claims.

Claims

1. An SDI transmission method for push-scan imaging data of a remote sensing camera based on FPGA, characterized by: The method is implemented by the following steps: Step 1: FPGA decodes the received push-scan imaging data of the remote sensing camera to obtain data that meets the SDI transmission bandwidth; Step 2: The FPGA calculates the pixel rate of the received push-scan imaging data of the remote sensing camera to obtain a pixel rate matching the SDI transmission; Step 3, storing the data conforming to the SDI transmission bandwidth obtained in step 1 into the first FIFO buffer according to the pixel rate matching the SDI transmission in step 2; Step 4: The FPGA sequentially reads the data in the FIFO, fills the invalid data generated in the rate mismatch area during the transmission process according to the matching pixel rate with the SDI transmission, obtains the SDI data stream of the push-scan camera, and sends it to the external receiving end through the FPGA's high-speed transceiver; Step 5: The external receiving end receives data through the high-speed transceiver of the FPGA, and identifies the line number, frame validity, and line validity flag in the SDI data stream to obtain valid data; Step 6: The external receiving end transmits the valid data back to the FPGA for decoding and stores it in the second FIFO buffer, and reads it according to the original push-scan camera rate to restore it to the original remote sensing camera push-scan imaging data.

2. The SDI transmission method of push-scan imaging data of a remote sensing camera based on FPGA according to claim 1, characterized in that: The FPGA includes a data processing module, a 20-bit reassembly module, a first FIFO buffer, a construction frame encoding module, a high-speed transceiver and an SDI protocol module; The push-scan imaging data of the remote sensing camera generates an imaging data stream under the drive of the FPGA. The image signal is processed by the data processing module and reorganized into data of the SDI transmission bandwidth by the 20-bit reorganization module. The data of the SDI transmission bandwidth is cached in the first FIFO buffer and then transmitted to the construction frame encoding module for splicing and filling of the imaging data stream to construct two-dimensional frame data that complies with the SDI transmission protocol. The SDI protocol module sends the two-dimensional frame data to the external receiving end through a high-speed transceiver.

3. The SDI transmission method of push-scan imaging data of a remote sensing camera based on FPGA according to claim 2, characterized in that: The FPGA also includes a frame decoding module, a second FIFO buffer and a 20-bit decomposition and reorganization module; When the FPGA receives the remote sensing camera imaging data that complies with the SDI protocol and is transmitted back from the external receiving end, the imaging data stream is transmitted to the FPGA through the high-speed transceiver, and the imaging data stream is converted into two-dimensional frame data through the SDI protocol module. The two-dimensional frame data is extracted into valid data of the remote sensing push-broom camera through the constructed frame decoding module, and the data is cached through the second FIFO buffer and parsed into the remote sensing camera push-broom imaging data that complies with the remote sensing camera protocol through the 20-bit decomposition and reorganization module. The remote sensing camera push-broom imaging data is parsed according to the original data rate to obtain an imaging data stream that complies with the rate and bandwidth of the remote sensing camera.

4. The SDI transmission method of push-scan imaging data of a remote sensing camera based on FPGA according to claim 3, characterized in that: Data is transmitted between the FPGA and the external receiving end through the high-speed serial interface of the high-speed transceiver; the FPGA transmits the serial signal to the external receiving end through a coaxial cable via a BNC physical interface and an SDI driver; the external receiving end transmits the push-scan imaging data of the remote sensing camera that complies with the SDI protocol to the FPGA through an SDI equalizer and a BNC physical interface.

5. The SDI transmission method of push-scan imaging data of a remote sensing camera based on FPGA according to claim 1, characterized in that: The first FIFO buffer and the second FIFO buffer are both asynchronous clock FIFOs.

6. The SDI transmission method of push-scan imaging data of a remote sensing camera based on FPGA according to claim 1, characterized in that: According to the calculated value of the clock match, the FIFO depth is designed, and the FIFO width is equal to the sum of the widths of the frame valid FVAL, the row valid LVAL, and the pixel data DATA.

7. The SDI transmission method of push-scan imaging data of a remote sensing camera based on FPGA according to any one of claims 1 to 6, characterized in that: In step 4, the frame encoding module inside the FPGA outputs the SDI data stream of the push-scan camera. The specific process is as follows: The input of the frame coding module is SDI data clock, frame coding module enable and asynchronous clock FIFO output data, and the total number of frames of the frame coding module is set to R, the total number of frames is set to L, the starting column is set to L0, the column width is set to L1, the starting line is set to R0, and the row width is set to R1; the output of the frame coding module is module data valid flag and SDI data stream; When the frame coding module is enabled, the number of columns and rows of the frame coding module starts to count, with the total number of frames and the total number of columns as the maximum value, and both start with 0. When the total number of columns is equal to the maximum value, the number of rows is increased by one and the number of columns is reset to zero. When the number of columns Yn is greater than L0 and less than L1+L0, and the number of rows is greater than R0 and less than R0+R1, the data valid flag is high, corresponding to the valid video in the SDI video transmission frame; at this time, if the asynchronous clock FIFO read enable is high, the data in the asynchronous clock FIFO is read, and the data is constructed as a 20-bit format remote sensing camera valid data output SDI video stream. If the read enable of the asynchronous clock FIFO is low at this time, the data in the SDI video stream is 3FFFFh, and the data valid flag is low at other times; the data in the SDI video stream is auxiliary data, and the SDI video stream is sent to the SDI driver through the SDI protocol module of the FPGA and the high-speed transceiver, generating an SDI format data signal and transmitting it to the external receiving end.

Images