Double-flow image transmission system and method based on high-speed SerDes interface

Abstract

The invention discloses a double-flow image transmission system and method based on a high-speed SerDes interface, and belongs to the technical field of high-speed image transmission, the system comprises a sending end and a receiving end, an image source A and an image source B of the sending end do not need DDR cache synchronization, and image frames are segmented, interleaved and packaged through a dynamic time slice distribution mechanism and then sent to a single high-speed SerDes channel; and a receiving end collects data and then de-interleaves and recombines the recovered image. The data encapsulation comprises a frame synchronization head, a block synchronization head and the like for identification, the dynamic time slice distribution mechanism alternately or independently sends image blocks according to the image preparation condition, and the image adopts a double-cache mechanism to avoid data coverage. The method improves the channel bandwidth utilization rate, saves hardware resources, is good in compatibility, supports various data stream combinations, reduces frame delay, and is suitable for single-channel multi-stream and multi-channel multi-stream transmission.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of high-speed digital image transmission, in particular to a dual-flow image transmission system and method based on a high-speed SerDes interface, which is a method and system for simultaneously performing lossless and low-latency interlaced transmission of two independent image data flows on a single-channel high-speed SerDes (Serializer / Deserializer) physical link implemented in an FPGA. BACKGROUND

[0002] With the widespread application of high-resolution and high-frame-rate cameras and image sensors, multiple image data need to be transmitted simultaneously through limited high-speed SerDes channels to a backend processing unit. In existing high-speed SerDes transmission technologies, there is a problem of low bandwidth utilization. When multiple image data need to be transmitted, multiple high-speed SerDes channels are often occupied, which not only increases the wiring cost but also increases the complexity of the hardware interface. The use of multiple channels makes the hardware design more complicated, and in the data transmission process, due to the synchronization of multiple channels, etc., it may affect the stability and efficiency of transmission. In addition, in some existing designs, DDR cache processing is required when processing multiple data stream combinations, which will cause large frame delay and affect the real-time performance of image transmission. SUMMARY

[0003] To solve the problems of low bandwidth utilization of high-speed SerDes transmission, increased cost and complexity of multi-channel transmission, and large frame delay in existing technologies, the present application provides a dual-flow image transmission system and method based on a high-speed SerDes interface.

[0004] The technical solution adopted by the present application is as follows: A dual-flow image transmission system based on a high-speed SerDes interface, comprising a sending end and a receiving end, the sending end comprising image source A, image source B and a processing module, and the receiving end comprising a receiving processing module; the image sources A and B of the sending end do not require DDR cache synchronization, and directly perform segmentation and interlaced packaging of image frames through a dynamic time slice allocation mechanism and send them into a single high-speed SerDes physical channel; after being collected by the receiving end through a single high-speed SerDes physical channel, the image sources A and B are recovered through deinterlacing and frame recombination.

[0005] The data encapsulation format includes frame synchronization headers and image block synchronization headers of images A and B; the image A frame synchronization header is a fixed 4-byte character 0x55aa_5a5a, which is transmitted at the beginning of each frame of the image A; the image A block synchronization header is a fixed 4-byte character 0x55aa_5a01, which is transmitted whenever the transmission of an A image block starts; the A image block is N bytes, which can be configured according to the SerDes bandwidth parameter; the block separation flag is a fixed 4-byte character 0x55aa_aa55, which is transmitted between every two image blocks; the image B frame synchronization header is a fixed 4-byte character 0xaa55_a5a5, which is transmitted at the beginning of each frame of the image B; the image B block synchronization header is a fixed 4-byte character 0xaa55_a501, which is transmitted whenever the transmission of a B image block starts; the B image block is M bytes, which can be configured according to the SerDes bandwidth parameter; and the data invalid character is 0x5555_aaaa, which is transmitted in the data invalid part between the frame synchronization headers and the block synchronization headers.

[0006] The dynamic time slice allocation mechanism divides the transmission time into small time slices, each of which transmits a fixed-size data block; when both the image A and the image B are ready, the A image blocks and the B image blocks are alternately sent; when only the image A is ready, the A image blocks are sent, and the B image blocks are not sent; and when only the image B is ready, the B image blocks are sent, and the A image blocks are not sent.

[0007] Both the image A and the image B use a double-buffering mechanism, the image A has two buffers buf: buf_a0 and buf_a1, and the image B has two buffers buf: buf_b0 and buf_b1; the two buffers buf of the same image are alternately used, when the image A buffer buf_a0 is selected, the transmission data serdes_tx_data of the image A is taken from the buf_a0, and the to-be-transmitted data data_A of the image A is buffered to the buf_a1; when the image A buffer buf_a1 is selected, the transmission data serdes_tx_data of the image A is taken from the buf_a1, and the to-be-transmitted data data_A of the image A is buffered to the buf_a0; and the data buffering process of the image B is the same as that of the image A.

[0008] A dual-flow image transmission method based on a high-speed SerDes interface, comprising the following steps: Step 1. The image sources A and B of the sending end generate image data, which does not need to pass through DDR buffer synchronization. Step 2. The sending end divides and interleaves the frames of images A and B through a dynamic time slice allocation mechanism to package; Step 3. The packaged data stream is sent into a single high-speed SerDes physical channel for transmission; Step 4. The receiving end collects the transmitted data stream through the single high-speed SerDes physical channel; Step 5. The receiving end performs deinterleaving and frame recombination on the collected data stream to recover images A and B.

[0009] In step 2, the data packaging format includes the frame synchronization header, block synchronization header, image block, block separation flag and data invalid character of images A and B. The image A frame synchronization header is a fixed 4-byte character 0x55aa_5a5a, which is transmitted at the beginning of each frame of image A. The image A block synchronization header is a fixed 4-byte character 0x55aa_5a01, which is transmitted whenever the transmission of an A image block starts. The A image block is N bytes, which can be configured according to the SerDes bandwidth parameters. The block separation flag is a fixed 4-byte character 0x55aa_aa55, which is transmitted between every two image blocks. The image B frame synchronization header is a fixed 4-byte character 0xaa55_a5a5, which is transmitted at the beginning of each frame of image B. The image B block synchronization header is a fixed 4-byte character 0xaa55_a501, which is transmitted whenever the transmission of a B image block starts. The B image block is M bytes, which can be configured according to the SerDes bandwidth parameters. The data invalid character is 0x5555_aaaa, which is transmitted in the data invalid part between the frame synchronization header and the block synchronization header.

[0010] In step 2, the specific operation of the dynamic time slice allocation mechanism is as follows: the transmission time is divided into small time slices, and each time slice transmits a fixed-size data block. When both image A and image B are ready, A image blocks and B image blocks are alternately sent. When only image A is ready, A image blocks are sent and B image blocks are not sent. When only image B is ready, B image blocks are sent and A image blocks are not sent.

[0011] The data buffer of the images A and B adopts a double buffer mechanism, that is, both images A and B have two buffers buf, and the two buffers buf of the same image are alternately used to avoid data buffer overwrite errors. The application is suitable for single-channel multi-stream and multi-channel multi-stream (the number of streams is greater than the number of channels) image transmission, and only needs to ensure that the SerDes bandwidth is greater than the total data stream bandwidth.

[0012] The application adopts the above technical scheme, and has the following beneficial effects: 1. Single channel can realize two-way or even multi-way image lossless transmission, single channel transmission of multiple data streams improves the utilization rate of channel bandwidth, saves hardware wiring and interface resources, reduces hardware cost and design complexity; 2. The dynamic time slice allocation mechanism has good compatibility, can support multiple data streams, arbitrary resolution and frame rate combinations, and has wide application range; 3. The data encapsulation structure is simple, the synchronization word and invalid padding word design has the characteristics of low power consumption and low complexity, and is easy to implement in FPGA; 4. Data stream processing does not require DDR cache synchronization, which can effectively reduce frame delay and ensure the real-time performance of image transmission. BRIEF DESCRIPTION OF DRAWINGS

[0013] Figure 1 The figure is a schematic diagram of the dual-flow image transmission system architecture of the application. Figure 2 The figure is a schematic diagram of the data encapsulation format of the application. DETAILED DESCRIPTION

[0014] The application will be described in detail below in combination with the drawings.

[0015] The dual-flow image transmission system based on high-speed SerDes interface of the application includes a sending end and a receiving end. The sending end includes image source A, image source B and corresponding processing modules, and the receiving end includes corresponding receiving processing modules. In actual application, the image sources A and B of the sending end can be image acquisition devices with different resolutions and different frame rates. The dynamic time slice allocation mechanism will determine the sending time of image block A and image block B. When image block A and image block B are ready for data output at the same time, image block A and image block B are sent alternately. When only image A is ready, image block A is sent and image block B is not sent. When only image B is ready, image block B is sent and image block A is not sent. Image block A and image block B are separated by block separation flag 0x55aa_aa55.

[0016] The pseudo code is as follows: while (trans_active) { if (imgA_ready && imgB_ready) { send_imgblock(A); / / Send image block A send_imgblock(B); / / Send image block B } else if (imgA_ready) { send_imgblock(A); / / A image exclusive time slot send_null_header(); } else if (imgB_ready) { send_null_header(); send_block(B); / / B image exclusive time slot } } 。

[0017] In the data encapsulation process, the beginning of each image frame is added with the corresponding frame synchronization header, and the frame synchronization header of image A is 0x55aa_5a5a, and the frame synchronization header of image B is 0xaa55_a5a5. The beginning of each image block is added with a block synchronization header, and the block synchronization header of image A is 0x55aa_5a01, and the block synchronization header of image B is 0xaa55_a501. The size N of the A image block and the size M of the B image block can be configured according to the bandwidth parameters of the SerDes to achieve the best transmission effect.

[0018] When the image blocks A and B are ready for data output at the same time, the A image block and the B image block are alternately sent. When only image A is ready, the A image block is sent, and the B image block is not sent. When only image B is ready, the B image block is sent, and the A image block is not sent.

[0019] The double buffer mechanism of images A and B is used to control the alternating use of the buffer through the clock signal clk and the buffer selection signals buf_sel_a and buf_sel_b when the hardware is implemented. When buf_sel_a is 1, the data is obtained from buf_a0, and the new data_A data is written into buf_a1 at the same time. When buf_sel_a is 0, the data is obtained from buf_a1, and the new data_A data is written into buf_a0, effectively avoiding the problem of data being overwritten in the buffer process.

[0020] The pseudo code is as follows: always @(posedge clk) begin if (buf_sel_a == 1) begin serdes_tx_data<= buf_a0; / / send A buffer buf_a0 buf_a1<= data_A; / / write A buffer buf_a1 end else begin serdes_tx_data<= buf_a1; / / Send A buffer buf_a1 buf_a0 <= data_A; / / Write data to buffer A, buf_a0 end.

[0021] The data caching process for image B is the same as that for image A.

[0022] After receiving the data stream, the receiving end distinguishes the data blocks of images A and B based on the frame synchronization header and block synchronization header. Then, it performs deinterleaving and frame reassembly to restore the segmented image blocks into complete image frames, thereby obtaining the original image sources A and B.

[0023] This invention is applicable not only to dual-stream image transmission, but also to single-channel multi-stream and multi-channel multi-stream (number of streams greater than number of channels) image transmission, as long as the SerDes bandwidth is greater than the total bandwidth of the data stream. Example

[0024] On the Xilinx Kintex-7 FPGA platform, two 1920×1080@60 Hz, 24-bit RGB images are transmitted simultaneously using a single 8 Gbps SerDes channel.

[0025] Each pixel is 24 bits, and the data volume per frame is approximately 1920 × 1080 × 24 bits ≈ 49.77 Mbit. After 8b / 10b encoding, the actual data size per frame is approximately 49.77 Mbit / 8 × 10 ≈ 62.21 Mbit. Single frame time: 16.67 ms; Required bandwidth: 62.21 Mbit / 16.67 ms ≈ 3.73 Gbps; Total bandwidth of the two channels: 7.46 Gbps < 8 Gbps, which meets the bandwidth requirements.

[0026] The sending end working process: 1. Divide each frame of image into several image blocks of 1920×3 bytes; 2. At the start of a frame, send a 4-byte frame synchronization header: 0x55aa_5a5a (stream A) or 0xaa55_a5a5 (stream B); 3. When neither of the two image streams is ready, send a 4-byte invalid padding word 0x5555_aaaa; 4. When only one image is ready, then send the image block data, and 4 bytes block sync head 0x55aa_5a01 or 0xaa55_a501 is inserted before the image block; 5. When both images are ready, send image block A and image block B alternately. That is, first send image block A (or B), and 4 bytes block sync head 0x55aa_5a01 (or 0xaa55_a501) is inserted before the image block; next time, send image block B (or A), and 4 bytes block sync head 0xaa55_a501 (or 0x55aa_5a01) is inserted before the image block; and so on; 6. 4 bytes block separation mark 0x55aa_aa55 is inserted between blocks; 7. Double buffer unit alternately writes and reads to avoid buffer conflict.

[0027] The working process of the receiving end is as follows: 1. Detect the frame sync head to determine the start of the frame; 2. Complete the deinterleaving according to the block sync head and the block separation mark; 3. Write the image block into the corresponding receiving buffer, and output after completing the whole frame reorganization; 4. If invalid padding words are detected, discard and do not process subsequently.

[0028] Experimental results After HDMI loopback test, both images are losslessly transmitted, the frame delay measurement value is 1.8us, which meets the requirements of industrial cameras and real-time image processing. Embodiment

[0029] On the basis of embodiment 1, the system is expanded to 4-way 1280x720@60Hz image transmission through 2-way 8Gbps SerDes channel, the method is the same, and only the dynamic time slice allocation strategy needs to be adjusted to stably operate.

[0030] The application realizes lossless and low-delay transmission of multi-way images on a single-channel SerDes through simple frame structure, dynamic time slice allocation and double buffer mechanism, overcomes multiple defects in the prior art, and can be widely applied in the fields of machine vision, medical imaging, vehicle-mounted camera, etc.

Claims

1. A dual-stream image transmission system based on a high-speed SerDes interface, characterized in that: The system includes a transmitter and a receiver. The transmitter includes image source A, image source B, and a processing module. The receiver includes a receiving and processing module. Image sources A and B at the transmitter do not require DDR cache synchronization. Instead, they directly segment and interleave image frames using a dynamic time-slice allocation mechanism and send them into a single high-speed SerDes physical channel. After acquisition through the single high-speed SerDes physical channel, the receiver recovers image sources A and B through deinterleaving and frame reconstruction.

2. The dual-stream image transmission system based on a high-speed SerDes interface according to claim 1, characterized in that: The data encapsulation format includes frame synchronization headers and image block synchronization headers for images A and B. The frame synchronization header for image A is a fixed 4-byte character set (0x55aa_5a5a), transmitted at the beginning of each frame of image A. The image block synchronization header for image A is a fixed 4-byte character set (0x55aa_5a01), transmitted whenever an image block of image A begins transmission. Image blocks of image A are N bytes long, configurable according to the SerDes bandwidth parameters. The block separator is a fixed 4-byte character set (0x55aa_aa55), transmitted between every two image blocks. The frame synchronization header for image B is a fixed 4-byte character set (0xaa55_a5a5), transmitted at the beginning of each frame of image B. The image block synchronization header for image B is a fixed 4-byte character set (0xaa55_a501), transmitted whenever an image block of image B begins transmission. Image blocks of image B are M bytes long, configurable according to the SerDes bandwidth parameters. Bandwidth parameters are configurable; invalid data characters are 0x5555_aaaa, and invalid data is transmitted in the invalid data portion between the frame synchronization header and the block synchronization header.

3. A dual-stream image transmission system based on a high-speed SerDes interface according to claim 1, characterized in that: The dynamic time-slice allocation mechanism divides the transmission time into small time slices, with each time slice transmitting a fixed-size data block. When both image A and image B are ready, image blocks A and B are transmitted alternately. When only image A is ready, image blocks A are transmitted, and image blocks B are not transmitted. When only image B is ready, image blocks B are transmitted, and image blocks A are not transmitted.

4. A dual-stream image transmission system based on a high-speed SerDes interface according to claim 1, characterized in that: Both images A and B employ a dual-buffering mechanism. Image A has two buffers: buf_a0 and buf_a1, and image B has two buffers: buf_b0 and buf_b1. The two buffers for the same image are used alternately. When buf_a0 is selected in image A's buffer, the data to be sent from image A (serdes_tx_data) is retrieved from buf_a0, and the data to be sent from image A (data_A) is buffered in buf_a1. When buf_a1 is selected in image A's buffer, the data to be sent from image A (serdes_tx_data) is retrieved from buf_a1, and the data to be sent from image A (data_A) is buffered in buf_a0. The data buffering process for image B is the same as that for image A.

5. A dual-stream image transmission method based on a high-speed SerDes interface, characterized in that... Includes the following steps: Step 1. Image sources A and B at the sending end generate image data without needing to synchronize via DDR cache; Step 2. The sending end uses a dynamic time slice allocation mechanism to segment and interleave frames of images A and B; Step 3. Send the encapsulated data stream into a single high-speed SerDes physical channel for transmission; Step 4. The receiving end collects the transmitted data stream through a single high-speed SerDes physical channel; Step 5. The receiving end performs deinterleaving and frame reconstruction on the acquired data stream to recover image sources A and B.

6. A dual-stream image transmission method based on a high-speed SerDes interface according to claim 5, characterized in that... In step 2, the data encapsulation format includes frame synchronization headers, block synchronization headers, image blocks, block separators, and invalid data characters for images A and B. The frame synchronization header for image A is a fixed 4-byte character set (0x55aa_5a5a), transmitted at the beginning of each frame of image A. The block synchronization header for image A is a fixed 4-byte character set (0x55aa_5a01), transmitted whenever an image block of A begins transmission. An image block of A is N bytes long, configurable according to the SerDes bandwidth parameters. The block separator is a fixed 4-byte character set (0x55aa_aa55), transmitted between every two image blocks. The frame synchronization header for image B is a fixed 4-byte character set (0xaa55_a5a5), transmitted at the beginning of each frame of image B. The block synchronization header for image B is a fixed 4-byte character set (0xaa55_a501), transmitted whenever an image block of B begins transmission. An image block of B is M bytes long, configurable according to the SerDes bandwidth parameters. The bandwidth parameter is configurable; the invalid data character is 0x5555_aaaa, which is transmitted between the invalid data portions of the frame synchronization header and the block synchronization header.

7. A dual-stream image transmission method based on a high-speed SerDes interface according to claim 5, characterized in that: In step 2, the dynamic time slice allocation mechanism operates as follows: the transmission time is divided into small time slices, and each time slice transmits a fixed-size data block; when both image A and image B are ready, image blocks A and B are sent alternately; when only image A is ready, image blocks A are sent and image blocks B are not sent; when only image B is ready, image blocks B are sent and image blocks A are not sent.

8. A dual-stream image transmission method based on a high-speed SerDes interface according to claim 5, characterized in that: The data caching of images A and B adopts a dual-caching mechanism, that is, each image A and B has two cache buffers. The two cache buffers of the same image are used alternately to avoid data overwriting errors during caching.

Images