Image data acquisition and transmission method and system with LVDS sampling clock phase self-calibration

Abstract

The invention relates to an image data acquisition and transmission method with LVDS sampling clock phase self-calibration. The method comprises: an FPGA chip configures registers in an image acquisition device through an SPI interface; a CIS of the image acquisition device acquires image data and transmits the data to an LVDS interface, and the image data output by the LVDS interface is automatically phase-adjusted by a clock phase calibration unit; parallel data in five data channels after clock phase adjustment are transmitted to a virtual BRAM of the FPGA chip for caching; the method performs clock phase calibration on each acquired image data before transmitting the data to a storage area for storage, thereby avoiding offset between a sampling clock and sampling data, and ensuring the stability and accuracy of the sampling data; and the transmission rate of the data after clock phase adjustment can reach 175MB / s, so that a large amount of image information can be transmitted in real time, and the timeliness of data transmission is ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of image data processing, and in particular to an image data acquisition and transmission method and system with LVDS sampling clock phase self-calibration. Background Art

[0002] In recent years, image acquisition and transmission systems have been widely used in various fields such as industry, military, and medicine, such as face recognition, remote monitoring, and videophones. There are many ways to acquire images, and cameras are usually used to acquire images in real time. In the process of image data acquisition and transmission, we usually have to consider the data transmission speed and the cacheable data capacity. The image data acquisition and transmission system in the prior art has the following main disadvantages: 1. The internal memory capacity of the system is small and cannot cache a large amount of image data; 2. The real-time bandwidth of the system is low and cannot meet the real-time high-bandwidth data transmission; 3. In order to meet the demand for caching a large amount of image data, an external memory needs to be added, which is costly.

[0003] FPGA is a programmable logic device, which is widely used in fields such as communication and video image processing. These applications require a large amount of BRAM in FPGA. When the BRAM capacity cannot meet the requirements, you may consider replacing a larger FPGA or expanding external storage devices such as SDRAM and DDR. However, expanding similar storage devices requires the support of FPGA and memory IP. For small FPGA devices or FPGA devices without memory hard IP, increasing storage will greatly increase system costs.

[0004] LVDS (Low Voltage Differential Signaling) interface, also known as RS-644 bus interface, is currently widely used in the field of video image processing. As the LVDS transmission rate continues to increase, the clock cycle occupied by each bit of the data signal continues to shorten, resulting in a decrease in the sampling accuracy of the sampling clock within the data valid clock cycle. In addition, the delay time of the data signal and the data sampling clock arriving at the receiving end may not be consistent, which may cause an offset between the sampling clock and the data, resulting in errors in the sampled data. Summary of the invention

[0005] The technical problem to be solved by the present invention is to provide an image data acquisition and transmission method and system with LVDS sampling clock phase self-calibration that can meet real-time high-bandwidth data transmission and avoid offset between sampling clock and sampling data, thereby improving the timeliness, stability and accuracy of data transmission.

[0006] The technical solution adopted by the present invention is an image data acquisition and transmission method with LVDS sampling clock phase self-calibration, the method comprising the following steps:

[0007] S1, the ARM processor transmits control instructions for configuring registers in the image acquisition device and collecting image data by the image acquisition device to the FPGA chip through the serial port. After receiving the control instructions, the FPGA chip configures the registers in the image acquisition device through the SPI interface;

[0008] S2. After the configuration is completed, the CIS of the image acquisition device acquires image data and transmits it to the LVDS interface. The image data output by the LVDS interface is automatically phase-adjusted by the clock phase calibration unit. The specific process of automatically phase-adjusting the image data by the clock phase calibration unit is as follows:

[0009] S2.1. Convert the differential data signals in the five data channels and the differential data signal in one clock channel output by the LVDS interface into single-ended data signals to obtain single-ended data signals of the five data channels and a single-ended data signal of the clock channel;

[0010] S2.2, converting the single-ended data signals in the five data channels into parallel data;

[0011] S2.3, select any one of the data channels, and adjust the clock phase of the parallel data in the data channel; then select the clock phase of the parallel data in the remaining data channels in turn, until the clock phase adjustment of the parallel data in the five data channels is completed; the process of adjusting the clock phase of the parallel data in the data channels is:

[0012] S2.31. Take the initial clock phase value of the parallel data in the data channel as the relatively stable phase value, sample the parallel data in the data channel according to the relatively stable phase value, and perform serial-to-parallel operations on the sampled data in a deserialization manner according to a set ratio to obtain data of several different characteristics;

[0013] S2.32, based on the relatively stable phase value, the relatively stable phase value is decreased, and the sampling clock phase is adjusted according to the phase value obtained by each decrease. After the clock phase is adjusted, the parallel data is sampled, and then the sampled data is serialized and operated in a deserialization manner according to the set ratio; until a certain data obtained by deserialization does not belong to one of the several data with different characteristics obtained in step S2.31, the relatively stable phase value stops decreasing, and the current decremented phase value is used as the critical value tap min ;

[0014] S2.33, based on the relatively stable phase value, the relatively stable phase value is incremented, and the sampling clock phase is adjusted according to the phase value obtained by each increment. After the clock phase is adjusted, the parallel data is sampled, and then the sampled data is serialized and operated in a deserialization manner according to the set ratio; until a certain data obtained by deserialization does not belong to one of the several data with different characteristics obtained in step S2.31, the relatively stable phase value stops increasing, and the current incremented phase value is used as the critical value tap max ;

[0015] S2.34, tap the critical value obtained min and critical value tap max Take the mean, that is: The average value obtained As the final sampling clock phase value;

[0016] S2.35, phase-shifting the sampling clock according to the final sampling clock phase value so that the sampling clock edge is at the center of the sampled data;

[0017] S2.4. The parallel data in the five data channels after the clock phase adjustment is transmitted to the virtual BRAM of the FPGA for caching.

[0018] The beneficial effects of the present invention are as follows: by adopting the above-mentioned image data acquisition and transmission method with LVDS sampling clock phase self-calibration, the method performs clock phase calibration on each acquired image data before transmitting it to the storage area for storage, thereby avoiding the offset between the sampling clock and the sampling data, and ensuring the stability and accuracy of the sampling data; and the transmission rate of the data after the clock phase adjustment can reach 175MB / s, which can realize real-time transmission of a large amount of image information and ensure the timeliness of data transmission.

[0019] Preferably, the virtual BRAM is a 64M SDRAM, which can cache large-capacity data information.

[0020] An image data acquisition and transmission system with LVDS sampling clock phase self-calibration, the system comprising: a power management unit, a system main control unit and an FPGA chip; the system main control unit comprises an ARM processor, the FPGA chip comprises an FPGA control module, a clock phase calibration unit and an FPGA storage module, the FPGA control module is connected to the ARM processor via a serial port, the FPGA control module is connected to an image acquisition device via an SPI interface, the ARM processor transmits control instructions for configuring registers in an image acquisition device and for the image acquisition device to acquire image data to the FPGA via the serial port, after receiving the control instructions, the FPGA configures the registers in the image acquisition device via the SPI interface; the clock phase calibration unit comprises A differential-to-single-ended module, a serial-to-parallel conversion module, a channel selection module, a phase alignment module and a delay adjustment module. The differential-to-single-ended module is connected to the LVDS interface in the image acquisition device. The differential-to-single-ended module is used to convert the differential data signals in the five data channels output by the LVDS interface and the differential data signals in one clock channel into single-ended data signals; the serial-to-parallel conversion module is used to convert the single-ended data signals in the five data channels obtained into parallel data; the channel selection module is used to select a data channel for phase adjustment, the phase alignment module is used to calculate the final sampling clock phase value, and the delay adjustment module is used to phase-shift the sampling clock according to the calculated final sampling clock phase value, so that the sampling clock edge is at the center of the sampling data.

[0021] Preferably, the FPGA storage module includes a virtual BRAM, a FIFO memory and a MIPI interface. The virtual BRAM is a 64M SDRAM. The virtual BRAM is used to cache the parallel data in the five data channels that have completed the clock phase adjustment. The virtual BRAM transmits the parallel data to the MIPI interface through the FIFO memory, and the MIPI interface then sends the parallel data to the ARM processor for data sorting operations. With this structure, large-capacity data information can be cached, and the caching and read-write operations of a large amount of image data can be realized. The module avoids the expansion of external memory, which not only saves system costs but also reduces development difficulty.

[0022] The above-mentioned image data acquisition and transmission system with LVDS sampling clock phase self-calibration is adopted. The system can perform clock phase calibration on each acquired image data before transmitting it to the storage area for storage, thereby avoiding the offset between the sampling clock and the sampling data, ensuring the stability and accuracy of the sampling data; and the transmission rate of the data after the clock phase adjustment can reach 175MB / s, which can realize real-time transmission of a large amount of image information and ensure the timeliness of data transmission. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 This is a system structure diagram of an image data acquisition and transmission system with LVDS sampling clock phase self-calibration according to the present invention;

[0024] Figure 2 A schematic diagram of the automatic phase adjustment of the image data output by the LVDS interface of the present invention through the clock phase calibration unit;

[0025] Figure 3 A schematic diagram of clock phase adjustment for parallel data in a data channel in the present invention;

[0026] Figure 4 It is a structural schematic diagram of the FPGA storage module in the present invention. DETAILED DESCRIPTION

[0027] The invention will be further described below with reference to the accompanying drawings and in combination with specific implementations, so that those skilled in the art can implement the invention with reference to the description. The protection scope of the invention is not limited to the specific implementations.

[0028] The present invention relates to an image data acquisition and transmission method with LVDS sampling clock phase self-calibration. Taking the acquisition and transmission of a piece of image data as an example, the method comprises the following steps:

[0029] S1, such as Figure 1 As shown, the ARM processor RK3399 transmits control instructions for configuring registers in the image acquisition device and collecting image data of the image acquisition device to the FPGA chip through the serial port. After receiving the control instructions, the FPGA chip configures the registers in the image acquisition device through the SPI interface;

[0030] S2. After the configuration is completed, the CIS of the image acquisition device acquires image data and transmits it to the LVDS interface. The image data output by the LVDS interface is automatically phase-adjusted by the clock phase calibration unit. The specific process of automatically phase-adjusting the image data by the clock phase calibration unit is as follows:

[0031] S2.1, such as Figure 2 As shown, the differential data signals in the five data channels and the differential data signal in one clock channel output by the LVDS interface are converted into single-ended data signals to obtain single-ended data signals of the five data channels and a single-ended data signal of the clock channel; for example, a pair of differential data lvds_data_p[0] and lvds_data_n[0] are converted into a single-ended data signal lvds_data[0];

[0032] S2.2, converting the single-ended data signals in the five data channels into parallel data;

[0033] S2.3, select any one of the data channels, and adjust the clock phase of the parallel data in the data channel; then select the clock phase of the parallel data in the remaining data channels in turn, until the clock phase adjustment of the parallel data in the five data channels is completed; this can avoid multi-channel phase adjustment at the same time, which occupies too many internal storage resources; the process of adjusting the clock phase of the parallel data in the data channel is:

[0034] S2.31, such as Figure 3 As shown, the data arrives before the clock, and the initial clock phase value of the parallel data in the data channel is taken as the relatively stable phase value. The parallel data in the data channel is sampled according to the relatively stable phase value, and the sampled data is serialized and parallelized in a set ratio deserialization manner to obtain a number of different data. For example, take an image with 200-bit data as an example, which is transmitted through five data channels respectively, each data channel transmits 40 bits, and the parallel data output by the data channel each time is set to 10 bits, in the form of 1111100000, then four consecutive times are required to complete 40 bits. The output of the data is 1111100000_1111100000_1111100000_1111100000. The 40-bit data is deserialized at a ratio of 1:8. There are ten features of the data that can be obtained, namely: 1111_1000, 1111_0000, 1110_0000, 1100_0001, 1000_0011, 0000_0111, 0000_1111, 0001_1111, 0011_1110, 0111_1100, i.e., data with ten different features.

[0035] S2.32, based on the relatively stable phase value, the relatively stable phase value is decreased, and the sampling clock phase is adjusted according to the phase value obtained by each decrease, and the parallel data is sampled after the clock phase is adjusted, and then the sampled data is serialized and operated in a deserialization manner according to the set ratio; until a certain data obtained by deserialization does not belong to one of the ten data with different characteristics obtained in step S2.31, the relatively stable phase value stops decreasing, and the current decremented phase value is used as the critical value tapmin;

[0036] S2.33, based on the relatively stable phase value, the relatively stable phase value is incremented, and the sampling clock phase is adjusted according to the phase value obtained by each increment. After the clock phase is adjusted, the parallel data is sampled, and then the sampled data is serialized and operated in a deserialization manner according to the set ratio; until a certain data obtained by deserialization is not one of the ten different data obtained in step S2.31, the relatively stable phase value stops increasing, and the phase value obtained by the current increment is used as the critical value tapmax;

[0037] S2.34, take the average of the critical value tapmin and the critical value tapmax, that is: The average value obtained As the final sampling clock phase value;

[0038] S2.35, phase shifting the sampling clock according to the final sampling clock phase value so that the sampling clock edge is located at the center of the sampled data; the above steps adjust the phase shift of the sampling clock so that the clock edge is located at the center of the data, and avoid the occurrence of metastable conditions while satisfying the clock bit alignment, thereby ensuring the stability of data transmission;

[0039] S2.4. The parallel data in the five data channels after the clock phase adjustment are transmitted to the virtual BRAM of the FPGA for caching. The virtual BRAM is a 64M SDRAM, which can cache large-capacity data information.

[0040] The process of transmitting the image data to the virtual BRAM of the FPGA for caching in the method of the present invention adopts the LVDS transmission mode with a transmission bandwidth of 175MB / s. In order to solve the offset problem between the sampling clock and the data, the present invention provides an LVDS sampling phase automatic adjustment function, adds a LVDS delay single channel, and realizes the adjustment of the relative phase of the clock and the data by adjusting the delay of the data and the clock, thereby realizing stable data sampling.

[0041] The above-mentioned image data acquisition and transmission method with LVDS sampling clock phase self-calibration is adopted. This method performs clock phase calibration on each acquired image data before transmitting it to the storage area for storage, thereby avoiding the offset between the sampling clock and the sampling data, ensuring the stability and accuracy of the sampling data; and the transmission rate of the data after the clock phase adjustment can reach 175MB / s, which can realize real-time transmission of a large amount of image information and ensure the timeliness of data transmission.

[0042] In step S2.3, the serial-to-parallel operation of the sampled data in accordance with the set ratio deserialization method is mainly to use a deserializer inside the FPGA chip to deserialize the parallel data. The set ratio is 1:8, and the strategy of exchanging area for speed is used, and the DDR (dual edge) working mode is adopted for deserialization.

[0043] An image data acquisition and transmission system with LVDS sampling clock phase self-calibration, such as Figure 1 As shown, the system includes: a power management unit, a system main control unit and an FPGA chip; the system main control unit includes an ARM processor, the FPGA chip includes an FPGA control module, a clock phase calibration unit and an FPGA storage module, the FPGA control module is connected to the ARM processor through a serial port, the FPGA control module is connected to the image acquisition device through an SPI interface, the ARM processor transmits control instructions for configuring registers in the image acquisition device and for the image acquisition device to acquire image data to the FPGA through the serial port, and after receiving the control instructions, the FPGA configures the registers in the image acquisition device through the SPI interface; the clock phase calibration unit includes a differential to single-ended module, a serial-to-parallel converter The invention relates to a method for manufacturing an image acquisition device comprising: a differential-to-single-ended module, a channel selection module, a phase alignment module and a delay adjustment module, wherein the differential-to-single-ended module is connected to the LVDS interface in the image acquisition device, and the differential-to-single-ended module is used to convert the differential data signals in the five data channels output by the LVDS interface and the differential data signals in one clock channel into single-ended data signals; the serial-to-parallel conversion module is used to convert the single-ended data signals in the five data channels obtained into parallel data; the channel selection module is used to select the data channel for phase adjustment, the phase alignment module is used to calculate the final sampling clock phase value, and the delay adjustment module is used to phase shift the sampling clock according to the calculated final sampling clock phase value, so that the sampling clock edge is at the center of the sampling data.

[0044] The FPGA storage module includes a virtual BRAM, a FIFO memory and a MIPI interface. The virtual BRAM is a 64M SDRAM. The virtual BRAM is clocked by a phase-locked loop PLL to control the operation of the memory. Then the user stores the image data decoded by LVDS in the virtual BRAM according to the method of reading and writing data in the internal BRAM, and performs read and write operations, thereby realizing the cache and read and write operations of a large amount of image data. This module avoids the expansion of external memory, which not only saves system costs but also reduces the difficulty of development; the virtual BRAM transmits parallel data to the MIPI interface through the FIFO memory, and the MIPI interface then sends the parallel data to the ARM processor for data sorting operations.

[0045] The FPGA storage module contains the device's own SDRAM, and its external interface is similar to the BRAM inside the FPGA, which is equivalent to mounting a virtual BRAM for caching the data information collected by the contact image sensor (CIS).

[0046] The above-mentioned image data acquisition and transmission system with LVDS sampling clock phase self-calibration is adopted. The system can perform clock phase calibration on each acquired image data before transmitting it to the storage area for storage, thereby avoiding the offset between the sampling clock and the sampling data, ensuring the stability and accuracy of the sampling data; and the transmission rate of the data after the clock phase adjustment can reach 175MB / s, which can realize real-time transmission of a large amount of image information and ensure the timeliness of data transmission.

Claims

1. An image data acquisition and transmission method with LVDS sampling clock phase self-calibration, characterized in that: The method comprises the following steps: S1, the ARM processor transmits control instructions for configuring registers in the image acquisition device and collecting image data by the image acquisition device to the FPGA chip through the serial port. After receiving the control instructions, the FPGA chip configures the registers in the image acquisition device through the SPI interface; S2. After the configuration is completed, the CIS of the image acquisition device acquires image data and transmits it to the LVDS interface. The image data output by the LVDS interface is automatically phase-adjusted by the clock phase calibration unit. The specific process of automatically phase-adjusting the image data by the clock phase calibration unit is as follows: S2.

1. Convert the differential data signals in the five data channels and the differential data signal in one clock channel output by the LVDS interface into single-ended data signals to obtain single-ended data signals of the five data channels and a single-ended data signal of the clock channel; S2.2, converting the single-ended data signals in the five data channels into parallel data; S2.3, select any one of the data channels, and adjust the clock phase of the parallel data in the data channel; then select the clock phase of the parallel data in the remaining data channels in turn, until the clock phase adjustment of the parallel data in the five data channels is completed; the process of adjusting the clock phase of the parallel data in the data channels is: S2.

31. Take the initial clock phase value of the parallel data in the data channel as the relatively stable phase value, sample the parallel data in the data channel according to the relatively stable phase value, and perform serial-to-parallel operations on the sampled data in a deserialization manner according to a set ratio to obtain data of several different characteristics; S2.32, based on the relatively stable phase value, the relatively stable phase value is decreased, and the sampling clock phase is adjusted according to the phase value obtained by each decrease. After the clock phase is adjusted, the parallel data is sampled, and then the sampled data is serialized and operated in a deserialization manner according to the set ratio; until a certain data obtained by deserialization does not belong to one of the several data with different characteristics obtained in step S2.31, the relatively stable phase value stops decreasing, and the current decremented phase value is used as the critical value tap min ; S2.33, based on the relatively stable phase value, the relatively stable phase value is incremented, and the sampling clock phase is adjusted according to the phase value obtained by each increment. After the clock phase is adjusted, the parallel data is sampled, and then the sampled data is serialized and operated in a deserialization manner according to the set ratio; until a certain data obtained by deserialization does not belong to one of the several data with different characteristics obtained in step S2.31, the relatively stable phase value stops increasing, and the current incremented phase value is used as the critical value tap max ; S2.34, tap the critical value min and critical value tap max Take the mean, that is: The average value obtained As the final sampling clock phase value; S2.35, phase-shifting the sampling clock according to the final sampling clock phase value so that the sampling clock edge is at the center of the sampled data; S2.

4. The parallel data in the five data channels after the clock phase adjustment is transmitted to the virtual BRAM of the FPGA for caching.

2. The image data acquisition and transmission method with LVDS sampling clock phase self-calibration according to claim 1, characterized in that: The virtual BRAM is 64M SDRAM.

3. An image data acquisition and transmission system with LVDS sampling clock phase self-calibration, used to implement the image data acquisition and transmission method with LVDS sampling clock phase self-calibration according to any one of claims 1 to 2, the system comprising: Power management unit, system main control unit and FPGA chip; The system main control unit includes an ARM processor, the FPGA chip includes an FPGA control module, a clock phase calibration unit and an FPGA storage module, the FPGA control module is connected to the ARM processor via a serial port, the FPGA control module is connected to the image acquisition device via an SPI interface, the ARM processor transmits control instructions for configuring registers in the image acquisition device and for the image acquisition device to acquire image data to the FPGA via a serial port, and after receiving the control instructions, the FPGA configures the registers in the image acquisition device via the SPI interface; the clock phase calibration unit includes a differential to single-ended module, a serial-to-parallel conversion module, a channel selection module, and a phase alignment module and a delay adjustment module, wherein the differential-to-single-ended module is connected to the LVDS interface in the image acquisition device, and the differential-to-single-ended module is used to convert the differential data signals in the five data channels output by the LVDS interface and the differential data signals in one clock channel into single-ended data signals; the serial-to-parallel conversion module is used to convert the single-ended data signals in the five data channels obtained into parallel data; the channel selection module is used to select the data channel for phase adjustment, the phase alignment module is used to calculate the final sampling clock phase value, and the delay adjustment module is used to phase shift the sampling clock according to the calculated final sampling clock phase value, so that the sampling clock edge is at the center position of the sampling data.

4. The image data acquisition and transmission system with LVDS sampling clock phase self-calibration according to claim 3, characterized in that: The FPGA storage module includes a virtual BRAM, a FIFO memory and a MIPI interface. The virtual BRAM is a 64M SDRAM. The virtual BRAM is used to cache parallel data in five data channels that have completed clock phase adjustment. The virtual BRAM transmits the parallel data to the MIPI interface through the FIFO memory, and the MIPI interface then sends the parallel data to the ARM processor for data sorting operations.

Images