A Multi-Protocol High-Speed ​​Image Data Reception and Alignment Method and System Based on FPGA

Abstract

This invention discloses a multi-protocol high-speed image data reception and alignment method and system based on FPGA, belonging to the fields of electronic information and image processing technology. It receives serial data streams from image sensors, dynamically calls FPGA serial input primitives to convert serial data into parallel data, and supports low-voltage differential signals, mobile industrial processor interfaces, and serial low-voltage signal protocols. A configurable sliding window mechanism is used to compare synchronization codes in real time and lock byte boundaries. By identifying the arrival time difference of data from each channel, hardware-level delay compensation is used to achieve multi-channel phase alignment, and data arriving early from other channels is compensated using a first-level register buffer. The image data's frame header and tail information is parsed to reconstruct the original image matrix and generate a video timing signal. This invention solves the problems of poor compatibility and high resource consumption in multi-protocol high-speed image data reception.

Description

Technical Field

[0001] This invention relates to the field of electronic information and image processing technology, specifically to a method and system for receiving and aligning high-speed image data based on FPGA and multiple protocols. Background Technology

[0002] With the widespread application of high-resolution, high-frame-rate CMOS image sensors in fields such as industrial vision and autonomous driving, reliable reception of their high-speed serial data output (such as LVDS, MIPI, and SLVS) has become a critical challenge. Existing FPGA-based receiving solutions have significant drawbacks:

[0003] Dedicated chip solution: Relies on external serial conversion chips, resulting in high system complexity, increased cost, poor flexibility, and difficulty in adapting to multi-protocol sensors.

[0004] FPGA native solution: Byte alignment mechanism relying on BITSLIP: In Xilinx's new FPGAs (such as the UltraScale series), the ISERDESE3 primitive has removed the BITSLIP function, causing the traditional alignment method to fail and resulting in poor platform compatibility.

[0005] The cumbersome multi-channel synchronization mechanism: It generally uses FIFO buffer or IDELAY delay line for channel alignment, which consumes a lot of valuable BRAM and logic resources, and introduces uncertain delays and timing jitter, making debugging complicated.

[0006] Official IP core solutions, such as Xilinx's MIPICSI-2 IP, are "black box" designs that cannot be customized or modified, and they support only one protocol, making them unsuitable for scenarios with heterogeneous sensors such as LVDS and SLVS.

[0007] Therefore, in order to address the above problems, there is an urgent need for a multi-protocol high-speed image data reception and alignment method and system based on FPGA. Summary of the Invention

[0008] To address the shortcomings of existing technologies, this invention provides a method and system for receiving and aligning multi-protocol high-speed image data based on FPGA, which solves the problems of poor compatibility and high resource consumption in multi-protocol high-speed image data reception.

[0009] To achieve the above objectives, the present invention provides the following technical solution: a multi-protocol high-speed image data receiving and alignment system based on FPGA, comprising a multi-protocol serial receiving interface adapter module, a synchronization code-assisted general byte alignment module, a multi-channel synchronization alignment module, and an image data stream extraction and video timing reconstruction module; the multi-protocol serial receiving interface adapter module receives serial data streams from image sensors and dynamically calls FPGA serial input primitives to convert serial data into parallel data, supporting low-voltage differential signals, mobile industrial processor interfaces, and serial low-voltage signal protocols; the synchronization code-assisted general byte alignment module is connected to the output of the multi-protocol serial receiving interface adapter module, and uses a configurable sliding window mechanism to compare synchronization codes in real time and lock byte boundaries; the multi-channel synchronization alignment module is connected to the output of the byte alignment module, and achieves multi-channel phase alignment by identifying the arrival time difference of data in each channel and using hardware-level delay compensation; the image data stream extraction and video timing reconstruction module is connected to the output of the channel alignment module, and is used to parse the frame header and frame tail information of the image data, reconstruct the original image matrix, and generate a video timing signal.

[0010] Furthermore, the multi-protocol serial receiver interface adaptation module includes a primitive selection unit, a clock generation unit, and a data conversion unit; the primitive selection unit is used to dynamically call FPGA serial input primitives, including IDDR primitives, ISERDESE2 primitives, and ISERDESE3 primitives, and select the corresponding instance according to the image sensor protocol type; the clock generation unit generates a source-synchronous clock network through a clock buffer and a frequency divider; the data conversion unit is used to convert serial signals into parallel data and output an eight-bit parallel data stream.

[0011] Furthermore, the synchronization code-assisted general byte alignment module includes a 40-bit sliding window register, an eight-way parallel comparator, a boundary flag latch, and a data selector; the 40-bit sliding window register is used to continuously buffer the deserialized parallel data stream; the eight-way parallel comparator performs parallel comparison between the data in the window and a 32-bit configurable synchronization code, the synchronization code content being dynamically set through the AXI-Lite interface; the boundary flag latch is triggered when 32 consecutive bits of data match a preset synchronization code, locking the current byte boundary position; the data selector selects the corresponding eight bits of data for output based on the boundary flag.

[0012] Furthermore, the multi-channel synchronization alignment module includes a phase detection unit, a data compensation unit, and an enable control unit; the phase detection unit identifies the latest arriving channel through a global data validity signal; the data compensation unit uses a first-level register buffer to achieve phase compensation for data arriving early in the channel; the enable control unit constrains the difference in the trace length of each channel's printed circuit board to no more than 1.0 centimeters based on the physical propagation delay characteristics; the enable control unit outputs a channel alignment enable signal only when all channel data is valid.

[0013] Furthermore, the image data stream extraction and video temporal reconstruction module extracts the synchronization signal by parsing the SAV or EAV identifier and combines it with the output of the front-end alignment module to realize image frame reconstruction.

[0014] Furthermore, it also includes a device adaptive configuration module, which includes an identifier reading unit, a mapping query unit, and a register configuration unit; the identifier reading unit reads the image sensor device identifier through the I2C bus or SPI bus; the mapping query unit queries the pre-stored identifier-protocol mapping table to obtain configuration parameters; the register configuration unit writes the parameters into the FPGA control register through the AXI-Lite bus.

[0015] Furthermore, the logic depth of the byte alignment module is no more than five levels, and the multi-channel synchronous alignment module achieves alignment through a physical delay compensation mechanism, without relying on the FIFO or IDELAY resources of the FPGA.

[0016] Furthermore, it also includes a system control and anomaly feedback module, which monitors the alignment status in real time and dynamically adjusts the synchronization code comparison parameters or phase compensation parameters based on error statistics to form a closed-loop optimization system.

[0017] The FPGA-based high-speed image data reception and alignment method, applying the aforementioned FPGA-based high-speed image data reception and alignment system, includes the following steps: receiving a serial data stream from an image sensor; dynamically calling FPGA serial input primitives to convert the serial data into parallel data, supporting low-voltage differential signals, mobile industrial processor interfaces, and serial low-voltage signal protocols; using a configurable sliding window mechanism to compare synchronization codes in real time and lock byte boundaries; identifying the arrival time difference of data from each channel and using hardware-level delay compensation to achieve phase alignment of multiple channels, compensating for data arriving early in a channel using a first-level register buffer; parsing the frame header and tail information of the image data, reconstructing the original image matrix, and generating a video timing signal.

[0018] The present invention has the following beneficial effects:

[0019] This FPGA-based high-speed image data reception and alignment method and system, through dynamic invocation of IDDR, ISERDESE2, and ISERDESE3 primitives, designs a unified receiving architecture, shielding underlying hardware differences. This allows the same design to be seamlessly ported to different generations of FPGAs, such as Xilinx 7 series and UltraScale, and simultaneously supports multiple physical layer protocols such as LVDS, MIPI, and SLVS. The proposed synchronization code-assisted general byte alignment algorithm replaces BITSLIP, employing a 40-bit sliding window and 8-way parallel comparators with a logic depth not exceeding 5 levels, achieving single-cycle byte locking with extremely low resource consumption (approximately 68 LUT + 56 FF per channel). An innovative lightweight multi-channel alignment mechanism utilizes PCB trace constraints and a first-level register cache instead of FIFO / IDELAY, achieving hardware-level, zero-jitter channel synchronization without consuming any BRAM or dedicated delay line resources. The device adaptive configuration module enables plug-and-play sensor functionality. The system control and anomaly feedback modules form a closed-loop optimization, dynamically adjusting parameters to cope with environmental changes, improving the system's long-term stability and robustness.

[0020] Of course, any product implementing this invention does not necessarily need to achieve all of the advantages described above at the same time. Attached Figure Description

[0021] Figure 1 This is a structural diagram of the FPGA-based multi-protocol high-speed image data receiving and alignment system of the present invention.

[0022] Figure 2 This is a flowchart of the multi-protocol high-speed image data reception and alignment method based on FPGA of the present invention. Detailed Implementation

[0023] This application embodiment achieves high-performance, low-resource data reception across FPGA platforms through an FPGA-based multi-protocol high-speed image data reception alignment method and system.

[0024] The overall concept of this application's embodiments is as follows:

[0025] By dynamically calling the FPGA's underlying deserialization primitives, compatibility with different physical protocols and chip platforms is achieved at the source. A general byte alignment algorithm based on configurable synchronization codes and parallel comparison is designed, and a sliding window is used to quickly lock byte boundaries. Hardware-level synchronization of multi-channel data is achieved by utilizing the physical delay characteristics of the PCB and combining it with a simplified first-level register buffer. While ensuring high performance, the system resource consumption and latency are significantly reduced, forming an efficient and reliable data receiving and alignment system.

[0026] Please see Figure 1This invention provides a technical solution: a multi-protocol high-speed image data receiving and alignment system based on FPGA, including a multi-protocol serial receiving interface adapter module, a synchronization code-assisted general byte alignment module, a multi-channel synchronization alignment module, and an image data stream extraction and video timing reconstruction module. The multi-protocol serial receiving interface adapter module receives serial data streams from image sensors and dynamically calls FPGA serial input primitives to convert serial data into parallel data, supporting low-voltage differential signals, mobile industrial processor interfaces, and serial low-voltage signal protocols. The synchronization code-assisted general byte alignment module is connected to the output of the multi-protocol serial receiving interface adapter module and uses a configurable sliding window mechanism to compare synchronization codes in real time and lock byte boundaries. The multi-channel synchronization alignment module is connected to the output of the byte alignment module and achieves multi-channel phase alignment by identifying the arrival time difference of data in each channel and using hardware-level delay compensation. The image data stream extraction and video timing reconstruction module is connected to the output of the channel alignment module and is used to parse the frame header and frame tail information of the image data, reconstruct the original image matrix, and generate a video timing signal.

[0027] Specifically, the overall architecture of this system includes a multi-protocol serial receiving interface adaptation module, a synchronization code-assisted general byte alignment module, a multi-channel synchronization alignment module, and an image data stream extraction and video temporal reconstruction module. These modules are connected in series to form a complete pipeline from data reception to temporal reconstruction.

[0028] The multi-protocol serial receiver interface adapter module is essentially a hardware protocol conversion layer. It adapts to different physical protocols by dynamically calling FPGA low-level primitives (such as IDDR, ISERDESE2, and ISERDESE3). The acquisition process is based on reading the identifier after the sensor is powered on and automatically matching it with the pre-stored protocol parameter table.

[0029] The core of the synchronization code-assisted general byte alignment module is a data boundary locking mechanism, which uses a configurable synchronization code (such as the 32-bit hexadecimal value 0xFF_00_00_AB) to compare the input data stream in real time. The synchronization code is dynamically configured by the host computer through the AXI-Lite interface, ensuring flexibility.

[0030] The multi-channel synchronous alignment module is essentially a phase compensator. It achieves hardware-level alignment by identifying the arrival time difference of data from each channel and using register buffering. Its parameters (such as trace length constraints ≤1.0cm) are preset based on the physical characteristics of the PCB layout.

[0031] The image data stream extraction and video temporal reconstruction module parses the frame structure (such as SAV / EAV identifiers) and reconstructs the image matrix. Its temporal signal generation depends on the valid data flags output by the front-end alignment module.

[0032] The modules are tightly coupled through data flow and control signals. For example, the parallel data output by the protocol adaptation module directly serves as the input to the byte alignment module; the boundary flags generated by the byte alignment module trigger the compensation logic of the channel alignment module; and finally, the timing reconstruction module relies on the coordinated output of all front-end modules to ensure frame integrity. This design makes any module indispensable—if the protocol adaptation module is omitted, the system cannot support multiple protocols; if the byte alignment module is removed, data misalignment will lead to frame loss.

[0033] Through modular pipeline design, the system achieves multi-protocol adaptation and high-frequency stable operation, while eliminating FIFO or IDELAY resources, significantly reducing power consumption and logic complexity.

[0034] Specifically, the multi-protocol serial receiver interface adapter module includes a primitive selection unit, a clock generation unit, and a data conversion unit. The primitive selection unit is used to dynamically call FPGA serial input primitives, including IDDR, ISERDESE2, and ISERDESE3 primitives, and select the corresponding instance according to the image sensor protocol type. The clock generation unit generates a source-synchronous clock network through a clock buffer and a frequency divider. The data conversion unit is used to convert serial signals into parallel data and output an eight-bit parallel data stream.

[0035] In this implementation, the primitive selection unit is essentially a hardware abstraction layer. The decision logic for dynamically calling the FPGA serial input primitives is based on sensor identifier lookup. For example, after reading the ID (0x0FC) of the Sony IMX252 sensor, the mapping table indicates the use of the ISERDESE3 primitive (suitable for the UltraScale platform). Primitive selection is implemented through a state machine, which loads the default configuration upon power-up and reconfigures it upon detecting a sensor change.

[0036] The clock generation unit generates a source synchronous clock network. For example, the 0.594Gbps serial rate of the LVDS interface corresponds to a 297MHz differential clock, which is then used by BUFGCE (Global Clock Buffer) and BUFGCE_DIV (Frequency Divider) to generate a 74.25MHz parallel clock. The clock parameters are determined by the protocol type and obtained through a lookup table.

[0037] The data conversion unit converts serial differential signals into 8-bit parallel data. For example, an LVDS signal is converted into a single-ended signal by an IBUFDS (differential input buffer) and then sent to the ISERDESE3 deserializer.

[0038] The primitive selection unit is crucial for cross-generational compatibility. IDDR is suitable for low-speed scenarios, ISERDESE2 supports 7-series FPGAs, and ISERDESE3 is optimized for the UltraScale platform. This dynamic calling shields the underlying hardware differences, ensuring that the same code can run on platforms such as Xilinx XC7Z035 and Zynq UltraScale+. The clock generation unit and data conversion unit work together: if the clock division is incorrect, the deserialized data will lose synchronization; if the primitive selection is mismatched, the data boundary cannot be locked. The module and the subsequent byte alignment module are connected through parallel data streams and valid signals to ensure seamless data transmission. This achieves plug-and-play and cross-platform compatibility for sensors, with a configuration time of <1μs, avoiding the cumbersome process of manually modifying IP cores in traditional solutions.

[0039] Specifically, the synchronization code-assisted general byte alignment module includes a 40-bit sliding window register, an eight-way parallel comparator, a boundary flag latch, and a data selector. The 40-bit sliding window register is used to continuously buffer the deserialized parallel data stream. The eight-way parallel comparator performs parallel comparisons of the data within the window with a 32-bit configurable synchronization code, the content of which is dynamically set via the AXI-Lite interface. The boundary flag latch is triggered when 32 consecutive bits of data match a preset synchronization code, locking the current byte boundary position. The data selector selects the corresponding eight bits of data for output based on the boundary flag.

[0040] In this implementation, the 40-bit sliding window register is essentially a data buffer. The theoretical basis for setting the width to 40 bits instead of 32 or 48 bits is that 32 bits are used to store the complete synchronization code (such as the 32-bit synchronization code 0xFF_00_00_AB), and the extra 8 bits serve as a "look-ahead window" to ensure that the complete synchronization code pattern can be captured at any offset position (such as an offset of 0-7 bits). For example, the window data pre_data[39:0] is shifted in real time to cover all possible byte boundaries.

[0041] The specific circuit structure of the 8-channel parallel comparator consists of 8 independent comparators that synchronously compare the matching status of sync_window[31:0], sync_window[32:1], and up to sync_window[39:8] with the synchronization code. The matching result generates boundary flags [7:0], with each channel corresponding to a possible byte offset position.

[0042] When 32 consecutive bits of data match the synchronization code (e.g., sync_window[31:0] matches completely), the latch immediately triggers to lock the current boundary; the data selector selects the correct 8 bits of data to output based on the flag.

[0043] A 40-bit window design is fundamental for high-speed alignment. Using only a 32-bit window requires multiple shifts to cover all boundaries, increasing latency; a 48-bit window would be wasteful of resources. An 8-channel parallel comparator implements single-cycle boundary locking, improving efficiency by 8 times compared to traditional serial search. The module's cohesion with its preceding and following modules is reflected in its input's parallel data stream dependent on the protocol adaptation module, and its output boundary flag directly driving the enable signal of the channel alignment module. Simplifying this module would require the system to rely on the FPGA's native BITSLIP mechanism, but BITSLIP is unavailable in ISERDESE3, leading to cross-platform failure.

[0044] Specifically, the multi-channel synchronous alignment module includes a phase detection unit, a data compensation unit, and an enable control unit; the phase detection unit identifies the latest arriving channel through the global data valid signal; the data compensation unit uses a first-level register buffer to achieve phase compensation for the data of the early arriving channel; the enable control unit constrains the difference in the trace length of each channel's printed circuit board to no more than 1.0 centimeters based on the physical propagation delay characteristics; the enable control unit outputs a channel alignment enable signal only when all channel data is valid.

[0045] In this implementation, the phase detection unit latches the enable state of each channel by the rising edge of the global data valid signal (such as byte_de) to identify the latest arriving channel. For example, in 16-channel image data, if the byte_de of channel 15 is the last to be valid, then it is set as the reference channel.

[0046] The data compensation unit uses a level-one register buffer (such as byte_data_r) to buffer data that arrives early in the channel, and selects delayed samples for output. The compensation logic is based on the time difference (Δt=ΔL×71.4ps / cm) converted from the PCB trace length difference (≤1.0cm), ensuring that the phase offset is ≤1 bit.

[0047] The enable control unit outputs the channel alignment enable signal lane_align_de only when all channel byte_de is valid (e.g., 8'hFF), to avoid misalignment caused by invalid data in some channels.

[0048] Physical constraints (trace length ≤ 1.0cm) are a prerequisite for the compensation mechanism—if the trace differences are too large, a single register buffer cannot compensate for the phase difference, requiring the introduction of a FIFO, which compromises the lightweight design. The module collaborates with the byte alignment module: the boundary markers output by byte alignment serve as the trigger condition for phase detection; if channel alignment fails, the timing reconstruction module will be unable to parse the complete frame. Furthermore, the module eliminates IDELAY resources, reducing logic depth through hardware-level compensation, consistent with the overall high-frequency target of the system. It achieves multi-channel zero-latency alignment, significantly reducing resource consumption compared to the FIFO scheme, and requires no external calibration.

[0049] Specifically, the image data stream extraction and video temporal reconstruction module extracts the synchronization signal by parsing the SAV or EAV identifier and combines it with the output of the front-end alignment module to realize image frame reconstruction.

[0050] In this implementation, the frame parsing unit is essentially a state machine that detects SAV / EAV identifiers (such as 0xFF000080) in the data stream in real time and generates line synchronization (HSYNC), field synchronization (VSYNC), and video data valid (VDE) signals. The parameters are determined by the sensor protocol; for example, in the SLVS protocol, SAV is located at the beginning of each line.

[0051] The RAW format rearrangement unit performs matrix reconstruction on Bayer format data (such as RGGB arrangement) and delineates the effective pixel area in conjunction with the synchronization signal. The rearrangement rules are configured through the AXI-Lite interface, supporting dynamic adaptation to different sensors.

[0052] The video stream encapsulation unit uses the Xilinx VideoIntoAXI-Stream IP core to package data into an AXI4-Stream standard stream (including signals such as TDATA and TVALID) for easy subsequent VDMA (Video Direct Memory Access) transmission.

[0053] The module is highly dependent on the quality of the front-end alignment. If byte or channel alignment is incorrect, SAV / EAV parsing will fail, leading to frame tearing. It works in conjunction with the adaptive configuration module: the sensor parameters loaded by the configuration module (such as a resolution of 2048×1544) directly determine the timing signal generation logic. This design ensures that the system only initiates reconstruction after front-end alignment is complete, avoiding resource waste.

[0054] Specifically, it also includes a device adaptive configuration module, which includes an identifier reading unit, a mapping query unit, and a register configuration unit. The identifier reading unit reads the image sensor device identifier through the I2C bus or SPI bus. The mapping query unit queries the pre-stored identifier-protocol mapping table to obtain configuration parameters. The register configuration unit writes the parameters into the FPGA control register through the AXI-Lite bus.

[0055] Specifically, it also includes a system control and anomaly feedback module, which monitors the alignment status in real time and dynamically adjusts the synchronization code comparison parameters or phase compensation parameters based on error statistics to form a closed-loop optimization system.

[0056] In this implementation, after power-on, the adaptive configuration reads the sensor ID (e.g., 0x0FC for Sony IMX252) via I2C / SPI, retrieves the protocol parameters from the mapping table, and writes them to the FPGA register via AXI-Lite. The mapping table is pre-stored on the ZynqPS and supports runtime updates.

[0057] The anomaly feedback module monitors the lane_invalid and sync_error flags, and dynamically adjusts parameters (such as synchronization code comparison sensitivity) after statistically analyzing error patterns. For example, if consecutive errors indicate timing jitter, the phase compensation threshold is automatically strengthened.

[0058] Adaptive configuration is the cornerstone of the system's plug-and-play capability—without this module, the FPGA must be manually recompiled every time a sensor is replaced. Closed-loop optimization improves long-term reliability: it relies on the status feedback of all front-end modules; if the byte alignment error rate increases, the optimization algorithm will adjust the synchronization code configuration, forming a positive loop.

[0059] Specifically, the logic depth of the byte alignment module is no more than five levels, and the multi-channel synchronous alignment module achieves alignment through a physical delay compensation mechanism, without relying on the FIFO or IDELAY resources of the FPGA.

[0060] In this implementation scheme, logical depth refers to the number of combinational logic layers along the data path. By optimizing the pipeline design, the depth of the byte alignment module is controlled to within 5 levels.

[0061] Level 1: Data shifting (40-bit window register update)

[0062] Second stage: Parallel comparison (8 comparators working synchronously).

[0063] Level 3: Flag Generation (Boundary Flag Logical Operations)

[0064] Level 4: Flag latch (stable output from trigger)

[0065] Level 5: Data Selection (Output 8-bit aligned data)

[0066] Static timing analysis verified that the path delay meets the requirements after setting a 350MHz clock constraint in Xilinx Vivado.

[0067] Shallow logic depth is a direct result of abandoning FIFO / IDELAY. Traditional solutions increase the depth to more than 10 levels due to resource allocation, limiting frequency improvement. This system achieves equivalent functionality through physical compensation and parallel comparison, ensuring that the total depth is controllable after the modules work together. For example, if the channel alignment module uses FIFO, its read / write control logic will increase by 3-4 levels, degrading overall performance.

[0068] Please see Figure 2A high-speed image data reception and alignment method based on FPGA, applying the aforementioned FPGA-based high-speed image data reception and alignment system, includes the following steps: receiving a serial data stream from an image sensor; dynamically calling FPGA serial input primitives to convert the serial data into parallel data, supporting low-voltage differential signals, mobile industrial processor interfaces, and serial low-voltage signal protocols; using a configurable sliding window mechanism to compare synchronization codes in real time and lock byte boundaries; identifying the arrival time difference of data from each channel and using hardware-level delay compensation to achieve multi-channel phase alignment, compensating for data arriving early in a channel using a first-level register buffer; parsing the frame header and tail information of the image data, reconstructing the original image matrix, and generating a video timing signal.

[0069] In summary, this application has at least the following effects:

[0070] The multi-protocol serial receive interface adapter module dynamically calls FPGA serial input primitives, compatible with multiple protocols such as low-voltage differential signals and mobile industrial processor interfaces. It can adapt to different image sensors without changing hardware, improving the system's versatility and adaptability. The synchronization code-assisted universal byte alignment module compares synchronization codes in real time with a configurable sliding window. Combined with the hardware-level delay compensation of the multi-channel synchronization alignment module, it quickly locks byte boundaries and eliminates channel phase differences. It does not rely on resources such as FIFOs and has low logic depth, ensuring the real-time performance and accuracy of high-speed image data reception alignment. The device adaptive configuration module automatically reads sensor identifiers and matches parameters. The system control module monitors the status in real time and optimizes parameters in a closed loop, reducing manual intervention and operational complexity. The image data stream extraction module efficiently reconstructs images and generates timing signals, significantly improving the overall efficiency, reliability, and adaptability of high-speed image data reception and processing, meeting the needs of high-resolution and high-frame-rate image transmission scenarios.

[0071] Those skilled in the art will understand that embodiments of the present invention can be provided as methods or systems. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0072] This invention is described with reference to flowchart illustrations and structural diagrams of methods and systems according to embodiments of the invention. It should be understood that the combination of each process and module in the flowchart and structural diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing device, generate instructions for implementing the process. Figure 1 One or more processes and structures Figure 1 A device for a function specified in one or more modules.

[0073] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and structures Figure 1 The function specified in one or more modules.

[0074] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and structures Figure 1 The steps of a specified function in one or more modules.

[0075] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0076] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A multi-protocol high-speed image data receiving alignment system based on FPGA, characterized in that, The application relates to a serial image data receiving system, which comprises a multi-protocol serial receiving interface adaptation module, a general byte alignment module assisted by a synchronization code, a multi-channel synchronization alignment module and an image data stream extraction and video timing reconstruction module. The multi-protocol serial receiving interface adaptation module is used for receiving a serial data stream from an image sensor and dynamically calling an FPGA serial input primitive to convert the serial data into parallel data, and supports low-voltage differential signal, mobile industry processor interface and serial low-voltage signal protocol. The multi-protocol serial receiving interface adaptation module comprises a primitive selection unit, a clock generation unit and a data conversion unit. The primitive selection unit is used for dynamically calling an FPGA serial input primitive, and comprises an IDDR primitive, an ISERDESE2 primitive and an ISERDESE3 primitive; and corresponding instances are selected according to the protocol type of the image sensor. The clock generation unit generates a source synchronization clock network through a clock buffer and a frequency divider. The data conversion unit is used for converting serial signals into parallel data and outputting eight-bit parallel data streams. The general byte alignment module assisted by the synchronization code is connected to the output end of the multi-protocol serial receiving interface adaptation module, adopts a configurable sliding window mechanism to compare the synchronization code in real time, and locks the byte boundary. The general byte alignment module assisted by the synchronization code comprises a forty-bit sliding window register, an eight-way parallel comparator, a boundary flag latch and a data selector. The forty-bit sliding window register is used for continuously buffering the parallel data stream after deserialization. The eight-way parallel comparator compares the data in the window with thirty-two-bit configurable synchronization codes in parallel, and the synchronization code content is dynamically set through an AXI-Lite interface. The boundary flag latch is triggered when the continuous thirty-two-bit data matches the preset synchronization code, and the current byte boundary position is locked. The data selector selects corresponding eight-bit data output according to the boundary flag. The multi-channel synchronization alignment module is connected to the output end of the byte alignment module, recognizes the time difference of data arrival of each channel, and realizes multi-channel phase alignment through hardware-level delay compensation. The multi-channel synchronization alignment module comprises a phase detection unit, a data compensation unit and an enable control unit. The phase detection unit recognizes the latest arrival channel through a global data valid signal. The data compensation unit realizes phase compensation for the data of the channel which arrives in advance through a first-order register buffer. The enable control unit restricts the length difference of printed circuit board wiring of each channel to not more than one zero centimeter based on the physical propagation delay characteristics. The enable control unit outputs a channel alignment enable signal only when the data of all channels are valid. The image data stream extraction and video timing reconstruction module is connected to the output end of the channel alignment module, is used for analyzing the frame header and frame tail information of the image data, reconstructs an original image matrix and generates a video timing signal.

2. The FPGA-based multi-protocol high-speed image data receiving alignment system according to claim 1, wherein, The image data stream extraction and video timing reconstruction module extracts a synchronization signal by analyzing SAV or EAV identifiers, and realizes image frame reconstruction in combination with the output of the front-end alignment module.

3. The FPGA-based multi-protocol high-speed image data receiving alignment system according to claim 1, wherein, The device adaptive configuration module comprises an identification reading unit, a mapping query unit and a register configuration unit. The identification reading unit reads the image sensor device identification through an I2C bus or an SPI bus; The mapping query unit queries a pre-stored identification-protocol mapping table to obtain configuration parameters; The register configuration unit writes the parameters into an FPGA control register through an AXI-Lite bus.

4. The FPGA-based multi-protocol high-speed image data receiving alignment system according to claim 1, wherein, The logical depth of the byte alignment module is not greater than five levels, and the multi-channel synchronous alignment module achieves alignment through a physical delay compensation mechanism and does not depend on the FIFO or IDELAY resources of the FPGA.

5. The FPGA-based multi-protocol high-speed image data receiving alignment system according to claim 1, wherein, Further comprising a system control and abnormal feedback module, which monitors the alignment state in real time and dynamically adjusts the synchronization code comparison parameters or phase compensation parameters according to error statistics to form a closed-loop optimization system.

6. The method for receiving and aligning multi-protocol high-speed image data based on FPGA, which applies the system for receiving and aligning multi-protocol high-speed image data based on FPGA according to any one of claims 1-5, characterized in that, The method comprises the following steps: Receiving a serial data stream from an image sensor, dynamically calling an FPGA serial input primitive to convert the serial data into parallel data, supporting low-voltage differential signal, mobile industry processor interface and serial low-voltage signal protocol; Using a configurable sliding window mechanism to compare the synchronization code in real time and lock the byte boundary; Through identifying the time difference of data arrival of each channel, using a hardware-level delay compensation to achieve multi-channel phase alignment, using a one-level register buffer to compensate the data of the channel that arrives in advance; Analyzing the frame header and frame tail information of the image data, reconstructing the original image matrix and generating a video timing signal.

Images