An underwater optical wireless real-time high-definition video transmission device based on FPGA

Abstract

The present invention discloses an underwater optical wireless real-time high-definition video transmission device based on FPGA, which includes a transmitting device and a receiving device, wherein the transmitting device includes a camera, an FPGA at the transmitting end, a digital-to-analog converter, a bias device and an LED in sequence; and the receiving device includes a focusing lens, a photodetector, a power amplifier, an attenuator, an analog-to-digital converter, an FPGA at the receiving end and a display in sequence. The device uses optical wireless communication and high-order digital signal modulation technology to improve the distance and quality of existing underwater video transmission, making the system transmission efficiency higher and the maintenance cost lower, and can achieve high-speed transmission of underwater video without using a network camera or Ethernet transmission.

Description

Technical Field

[0001] The invention belongs to the technical field of underwater wireless communication, and in particular relates to an underwater optical wireless real-time high-definition video transmission device based on FPGA in underwater wireless communication. Background Art

[0002] With the deepening of human activities such as marine resource exploration, oceanographic research and marine environment monitoring, the data transmitted between underwater devices has increased significantly. The rapid and real-time transmission of these data has put forward higher requirements on the communication capabilities of current underwater equipment. Although traditional underwater acoustic communication has a long transmission distance, it has a large transmission delay, low rate and serious multipath interference, which makes it difficult to meet the bandwidth requirements of real-time high-definition video image transmission. The strong conductivity of seawater limits the transmission distance of radio frequency radio. Low-frequency radio communication is often used in marine applications, but the large antenna size is not conducive to its deployment on miniaturized platforms. The use of submarine optical cable communication requires special submarine wiring, which is very expensive to implement. Underwater optical wireless communication has the advantages of large bandwidth, low delay, low energy consumption and high concealment, and is more competitive in high-speed data transmission.

[0003] Taking advantage of the low loss of underwater blue-green light, underwater optical wireless communication can support a longer transmission distance. According to the Chinese invention patent application CN109347556A published on February 15, 2019, a visible light-based underwater communication system is disclosed. The system uses an STC single-chip microcomputer to complete signal processing, which cannot meet the requirements of high-speed, high-bandwidth video transmission systems. In addition, existing underwater video transmission systems mostly use single-carrier low-order modulation formats, such as OOK modulation, with low system transmission rate and spectrum efficiency, and the transmission video resolution is 320*240 pixels. For details, see the reference: A. Al-Halafi, H.Makine Oubei, Boon S. Ooi, and B. Shihada, "Real-Time Video Transmission OverDifferent Underwater Wireless Optical Channels Using a Directly Modulated 520nm Laser Diode," J. OPT. COMMUN. NETW 9(10), 826-832(2017). In addition, many existing underwater video transmissions require the use of web cameras or Ethernet transmission. For details, see the reference: J. LI, B.YANG, D. YE, L. WANG, K. FU, J. PIAO, and Y. WANG, "A Real-Time, Full-Duplex Systemfor Underwater Wireless Optical Communication: Hardware Structure and OpticalLink Model," IEEE Access vol. 4, 109372-109387 (2020). The system transmission cost is high, an Ethernet card is required at the receiving end, and it is not easy to maintain. Summary of the invention

[0004] The main purpose of the present invention is to address the shortcomings of current underwater video transmission and provide an underwater optical wireless real-time high-definition video transmission device based on FPGA. High-order multi-carrier modulation is used to improve the video transmission rate. There is no need to use an Ethernet card or other equipment for reception. The display part of the device can be connected to any display or LCD display that supports HDMI at the HDMI interface of the FPGA. The size and power consumption of the device are smaller. The present invention is suitable for underwater robots performing underwater exploration and video transmission between underwater equipment platforms and other scenarios.

[0005] The technical solution of the present invention is as follows:

[0006] An underwater optical wireless real-time high-definition video transmission device based on FPGA includes a sending device and a receiving device, wherein the sending device includes a camera, an FPGA at the sending end, a digital-to-analog converter, a bias device and an LED in sequence; the receiving device includes a focusing lens, a photodetector, a power amplifier, an attenuator, an analog-to-digital converter, an FPGA at the receiving end and a display in sequence;

[0007] Among them, the camera completes video acquisition and signal generation, and the output digital video signal is transmitted to the FPGA at the sending end to complete signal processing, and then transmitted to the digital-to-analog converter for digital-to-analog conversion. The signal is connected to the bias device to couple the analog signal with DC, and the output end of the bias device is connected to the LED drive circuit to drive the LED to work and emit a light signal;

[0008] The focusing lens is used to converge the optical signal after being transmitted through the underwater channel. The photodetector converts the optical signal into a voltage signal. The voltage is adjusted by the amplifier and attenuator and then input into the analog-to-digital converter. The analog signal is converted into a digital signal and then input into the FPGA at the receiving end to complete the demodulation and recovery of the video data, and finally drives the display to play the video through the HDMI interface.

[0009] The signal processing part inside the FPGA of the transmitting end includes parallel-to-serial conversion of digital video signals, adding video frame header data for vertical synchronization of receiving end video, generation of long and short sequences, high-order multi-carrier modulation coding, pilot insertion and addition of cyclic prefix.

[0010] The FPGA at the transmitting end uses double rate synchronous dynamic memory (DDR) to store the collected video data, and FIFOs with different clocks are used on the reading and writing ends to complete burst reading and writing, realizing cross-clock domain data processing of different modules.

[0011] The FPGA at the transmitting end, the video frame header data is the preset 32-bit synchronization data, which is used for the vertical synchronization of the receiving end video. The short sequence is the 16-bit data with low correlation stored in 16 registers. The long sequence is the time domain data generated by Fourier inverse transform using the local frequency domain data, which is stored in a 1024*14-bit ROM and directly called by the main control module.

[0012] The FPGA at the receiving end performs group detection, video frame synchronization, symbol synchronization, cyclic prefix removal, Fourier transform, oversampling removal, frequency domain synchronization, channel estimation and channel frequency domain equalization on the received signal, and then restores the original digital signal through demodulation, then decodes the video signal and connects it to the RGB to HDMI module, and finally drives the display to work through the HDMI interface.

[0013] The FPGA at the receiving end determines the arrival of the data frame through packet detection and sends the currently receivedM Data and τ Received before time M The data are cross-correlated, and the relevant formula (1) is:

[0014]

[0015] in D k is the decision variable of the delay-dependent algorithm, R k is the signal energy at the receiving end, T k To delay the correlation value, the decision variable is greater than the preset threshold, that is, the decision signal arrives; the delay-related complex multiplication calculation implementation method in the FPGA is:

[0016]

[0017] in, U and V The complex data is input; in order to reduce hardware computing resources, three multipliers and five adders and subtractors are used to calculate Q real and Q imag , expressed as:

[0018]

[0019] in U real , U imag and V real , V imag Data U and V The real and imaginary parts of .

[0020] The FPGA at the receiving end performs channel estimation and equalization in the frequency domain, and the symbol after equalization is , which is expressed as follows:

[0021]

[0022] Where Z is the received video data symbol, is the conjugate number of the locally known long training sequence, the value is -1 or +1, Z seq It is the average value of two training symbols. When implemented in FPGA, the real part or imaginary part of the received long training symbol average value is inverted to obtain the conjugate value of the channel estimation, that is, the phase compensation factor. The phase compensation of the received video data symbol can be completed through channel equalization.

[0023] The digital-to-analog converter, analog-to-digital converter and FPGA are all connected using pin headers, the digital-to-analog converter / analog-to-digital converter, power amplifier, attenuator, bias device and LED drive circuit are all connected using radio frequency cables with SMA interfaces, and the DAC output and ADC input are connected using radio frequency cables with BNC interfaces.

[0024] The FPGA used is xc7z020clg400-2 series, the DAC used is AD9708, the ADC used is AD9238, the LED lamp is a blue / green light LED, the bias device used is ZFBT-4R2GW, and the detector used is avalanche photodiode Hamamatsu C12702 series.

[0025] Beneficial effects of the present invention:

[0026] (1) The present invention adopts optical wireless communication technology for video transmission, which has higher transmission rate and bandwidth than traditional underwater acoustic communication. Compared with underwater optical cable or cable communication, its implementation cost is greatly reduced, and high-definition video transmission can be achieved without Ethernet access, alleviating the problem of high transmission and maintenance costs.

[0027] (2) The receiving end of the present invention does not need to use an Ethernet card or other devices for receiving. The display part of the device can be connected to any display or LCD display that supports HDMI through the HDMI interface of the FPGA. The size and power consumption of the device are smaller. A single device is used to achieve high-speed video transmission and detection, which is of great significance to marine information research.

[0028] (3) The present invention adopts a high spectrum efficiency modulation method to effectively improve the video transmission rate. The video data uses a multi-level orthogonal amplitude phase modulation-orthogonal frequency division multiplexing modulation method (such as 16QAM-OFDM). Compared with the modulation methods such as non-return-to-zero on-off keying (NRZ-OOK) and phase shift keying (PSK) used in general video transmission systems, the use of high-order modulation formats effectively improves the transmission rate of video communication. Using a high-speed FPGA to perform high-order multi-carrier modulation (such as 16QAM-OFDM) on the video data can significantly improve the system communication rate under the condition of hardware DA / AD conversion rate, reaching a resolution of 1280*720 or above, and realizing high-definition video transmission. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 This is a structural block diagram of the underwater optical wireless real-time high-definition video transmission device based on FPGA of the present invention.

[0030] Figure 2 It is a schematic diagram of the system sending device of the present invention.

[0031] Figure 3 This is a schematic diagram of FPGA data processing in the sending device of the present invention.

[0032] Figure 4 It is a schematic diagram of the system receiving device of the present invention.

[0033] Figure 5 This is a schematic diagram of FPGA data processing in the receiving device of the present invention.

[0034] Figure 6 Schematic diagram of the relationship between system bit error rate and received optical power at different distances. DETAILED DESCRIPTION

[0035] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0036] Figure 1 This is a schematic diagram of an underwater optical wireless real-time high-definition video transmission device based on FPGA. The transmitting device includes a camera, an FPGA at the transmitting end, a digital-to-analog converter, a bias device, and a blue / green LED. The receiving device includes a focusing lens, a photodetector, a power amplifier, an attenuator, an analog-to-digital converter, an FPGA at the receiving end, and a display.

[0037] In the sending device, the camera completes video acquisition and signal generation, and the output digital video signal is transmitted to the FPGA at the sending end through the pin header to complete the relevant signal processing. The FPGA at the sending end is connected to the digital-to-analog converter through the pin header. The digital-to-analog converter connects the signal to the bias device through the BNC interface through the BNC-SMA RF cable. The output end of the bias device is connected to the LED drive circuit with a RF cable with an SMA interface to drive the LED to work in the linear region so that the LED luminous power meets the system transmission requirements.

[0038] In the receiving device, the focusing lens is used to converge the optical signal after being transmitted through the underwater channel. The photodetector converts the optical signal into a voltage signal. After passing through the amplifier and attenuator, the voltage is adjusted to -5~+5V and input to the RF interface of the analog-to-digital converter. After the analog signal is converted into a digital signal, it is input into the FPGA at the receiving end to complete the demodulation and recovery of the video data, and finally drives the display to play the video through the HDMI interface.

[0039] Figure 2 The schematic diagram of the sending device of the underwater optical wireless real-time high-definition video transmission device based on FPGA is shown in FIG. When working, the FPGA at the sending end first resets the camera and the double-rate synchronous dynamic random access memory DDR3, then the FPGA delays 20ms to power on the camera and communicates with the camera. 2C protocol communication completes the camera parameter configuration (such as completing the automatic exposure control and white balance of filtering light stripes, and completing the video pixel digital signal output after real-time shooting with a 25MHz pixel clock), so that the camera outputs pixel data in the RGB565 format. The FPGA at the sending end uses a "ping-pong structure" to store the input video data in the WrFIFO first, and then interacts with the double-rate synchronous dynamic random access memory DDR3 to complete the storage of one frame of the video signal, and then sends the data to the modulation module through the RdFIFO. Finally, the digital signal is converted into an analog signal through a digital-to-analog converter, and combined with the bias device to drive the LED to work.

[0040] Figure 3 It is the data processing part in the FPGA of the transmitting device, in which the top-level control module is responsible for the overall control of the entire modulation part, including the control of each bottom-level module and the generation of clock frequency division. The sequence generation module is a module for generating long and short sequences. The short sequence is directly generated by the local register, and the long sequence is generated by reading the time domain long sequence stored in the ROM. The video frame header generation module completes the header generation of the video frame, which is used for video frame synchronization at the receiving end. The data symbol generation module completes the orthogonal amplitude phase modulation (such as hexadecimal QAM modulation) of the video signal and the addition of the pilot. After that, the inverse Fourier transform module completes the conversion of data from the frequency domain to the time domain. The cyclic prefix addition and windowing module completes the addition of 256 data cyclic prefixes and windowing processing, and finally stores it in the FIFO output and transmits it to the digital-to-analog converter for transmission with a specific clock. For example, the video frame header data is the preset 32-bit synchronization data (32'hf0f0_f0f0); the short sequence is the 16-bit data with low correlation stored in 16 registers, and the long sequence is the time domain data generated by Fourier inverse transform using local frequency domain data, stored in a 1024*14-bit ROM. The sending device adds long and short sequences for the synchronization of the video line data of the receiving device, and adds video frame header data for the vertical synchronization of the video of the receiving device.

[0041] Figure 4 It is the receiving device of the system. The focusing lens converges the light beam after being transmitted through the underwater channel to the active area of ​​the photodetector (such as avalanche photodiode APD). The photodetector completes the conversion of optical signals to electrical signals. The electrical signals are connected to the amplifier through the RF line. The output voltage of the amplifier is 24V. The attenuator is adjusted to transmit the output voltage in the range of -5V~+5V to the analog-to-digital converter. The analog-to-digital converter converts the analog signal into a digital signal and inputs it into the FPGA at the receiving end. Then, the digital signal is demodulated, the video signal is restored, and the HDMI interface is driven in turn. Finally, the restored video information is played on the display.

[0042] Figure 5It is the FPGA data processing part of the receiving end. The digital signal input from the analog-to-digital converter to the FPGA is first switched by the WrFIFO to complete the switching of different clock domains and cache the data in the double-rate synchronous dynamic random access memory DDR3. The RdFIFO is responsible for sending the data to the demodulation module across the clock domain. First, the frame detection and packet detection are completed through the received signal energy detection algorithm to realize the packet detection of the incoming data and the frame synchronization of the video signal. The receiving end determines the arrival of the data frame through packet detection and compares the 16 data currently received with the received frame. τ The 16 data received before the time are cross-correlated, and the formula is:

[0043]

[0044] in D k is the decision variable of the delay-dependent algorithm, R k is the signal energy at the receiving end, T k To delay the correlation value, the decision variable is greater than the preset threshold, that is, the decision signal arrives. When multiple preset end thresholds are detected, the video signal is judged to be over. The symbol synchronization module uses the short sequence autocorrelation algorithm to complete the symbol synchronization output of the data. The cyclic prefix removal module completes the cyclic prefix removal of the long sequence and time domain data symbols. The Fourier transform realizes the conversion of the signal from the time domain to the frequency domain. In the frequency domain, the oversampling is removed and the 4 times oversampled data is removed to restore the complete frequency domain data. The frequency domain synchronization part is implemented by the frequency domain channel estimation and equalization, sampling frequency synchronization and phase tracking modules. The frequency domain channel estimation and equalization completes the frequency domain response correction of the channel, which can effectively compensate for the interference between subcarriers caused by channel multipath delay during underwater transmission. The symbol after frequency domain equalization is , which can be expressed as:

[0045]

[0046] Where Z is the received video data symbol, is the conjugate number of the locally known long training sequence, the value is -1 or +1, Z seq is the average value of two training symbols. When implemented in FPGA, it is only necessary to invert the real or imaginary part of the average value of the received long training symbol to obtain the conjugate value of the channel estimate (i.e., the phase compensation factor). The phase compensation of the received video data symbol can be completed through channel equalization.

[0047] The sampling frequency synchronization module 33 corrects the sampling period and phase errors caused by the different clock crystal phases of the DAC / ADC at the transmitting and receiving ends. In addition, as the number of transmitted symbols increases, the accumulation of system phase rotation misalignment will cause phase rotation errors in the demodulation module. The use of the phase tracking module can effectively compensate for the interference caused by phase noise, thereby effectively improving the accuracy of subsequent demodulation symbols.

[0048] The orthogonal amplitude phase demodulation module uses a hard decision method to make corresponding data decisions by comparing the real and imaginary parts of different thresholds. At the same time, the energy of the previous channel estimation module is used to dynamically scale the demodulated constellation diagram to restore the original video data. The error detection module completes the error detection of the corresponding data, and then the RGB to HDMI module completes the differential signal output in HDMI format. Finally, the HDMI is connected to the display to play the corresponding video. At different distances, the relationship between the bit error rate of the measurement system and the received optical power is shown in the figure. Figure 6 , the bit error rate is lower than the forward error correction decision threshold at different distances. The system can transmit to a distance of more than ten meters when it is lower than the forward error correction decision threshold.

[0049] The embodiments described above can be further combined or replaced, and the embodiments are only descriptions of preferred embodiments of the present invention, and do not limit the concept and scope of the present invention. Without departing from the design concept of the present invention, various changes and improvements made by ordinary technicians in this field to the technical solution of the present invention belong to the protection scope of the present invention. The protection scope of the present invention is given by the attached claims and any equivalents thereof.

Claims

1. An underwater optical wireless real-time high-definition video transmission device based on FPGA, characterized in that: The invention comprises a transmitting device and a receiving device, wherein the transmitting device comprises a camera, an FPGA at the transmitting end, a digital-to-analog converter, a bias device and an LED in sequence; the receiving device comprises a focusing lens, a photodetector, a power amplifier, an attenuator, an analog-to-digital converter, an FPGA at the receiving end and a display in sequence; Among them, the camera completes video acquisition and signal generation, and the output digital video signal is transmitted to the FPGA at the sending end to complete signal processing, and then transmitted to the digital-to-analog converter for digital-to-analog conversion. The signal is connected to the bias device to couple the analog signal with DC, and the output end of the bias device is connected to the LED drive circuit to drive the LED to work and emit a light signal; The focusing lens is used to gather the optical signal transmitted through the underwater channel. The photodetector converts the optical signal into a voltage signal. After the voltage is adjusted by the amplifier and attenuator, it is input to the analog-to-digital converter. After the analog signal is converted into a digital signal, it is input to the FPGA at the receiving end to complete the demodulation and recovery of the video data, and finally drives the display to play the video through the HDMI interface. The signal processing part inside the FPGA of the transmitting end includes parallel-to-serial conversion of digital video signals, adding video frame header data for vertical synchronization of receiving end video, generation of long and short sequences, high-order multi-carrier modulation coding, pilot insertion and addition of cyclic prefix; The FPGA at the transmitting end uses double rate synchronous dynamic memory (DDR) to store the collected video data, and uses FIFOs with different clocks to complete burst reading and writing at the reading and writing ends, realizing cross-clock domain data processing of different modules; The FPGA at the transmitting end, the video frame header data is the preset 32-bit synchronization data, which is used for the vertical synchronization of the receiving end video, the short sequence is the 16-bit data with low correlation stored in 16 registers, and the long sequence is the time domain data generated by the local frequency domain data through the inverse Fourier transform, which is stored in the 1024*14-bit ROM and directly called by the main control module; The FPGA at the receiving end performs group detection, video frame synchronization, symbol synchronization, cyclic prefix removal, Fourier transform, oversampling removal, frequency domain synchronization, channel estimation and channel frequency domain equalization on the received signal, and then restores the original digital signal through demodulation, then decodes the video signal and connects it to the RGB to HDMI module, and finally drives the display to work through the HDMI interface; The FPGA at the receiving end determines the arrival of the data frame through packet detection and sends the currently received M Data and τ Received before time M The data are cross-correlated, and the relevant formula (1) is: ； in d k is the decision variable of the delay-dependent algorithm, R k is the signal energy at the receiving end, T k To delay the correlation value, the decision variable is greater than the preset threshold, that is, the decision signal arrives; the delay-related complex multiplication calculation implementation method in the FPGA is: ； in, U and V The complex data is input; in order to reduce hardware computing resources, three multipliers and five adders and subtractors are used to calculate Q real and Q imag , expressed as: ； in U real , U imag and V real , V imag Data U and V The real and imaginary parts of The FPGA at the receiving end performs channel estimation and equalization in the frequency domain, and the symbol after equalization is , which is expressed as follows: ； Where Z is the received video data symbol, is the conjugate number of the locally known long training sequence, the value is -1 or +1, Z seq It is the average value of two training symbols. When implemented in FPGA, the real part or imaginary part of the received long training symbol average value is inverted to obtain the conjugate value of the channel estimation, that is, the phase compensation factor. The phase compensation of the received video data symbol can be completed through channel equalization.

2. The FPGA-based underwater optical wireless real-time high-definition video transmission device according to claim 1, characterized in that: The digital-to-analog converter, analog-to-digital converter and FPGA are all connected using pin headers, the digital-to-analog converter / analog-to-digital converter, power amplifier, attenuator, bias device and LED drive circuit are all connected using radio frequency cables with SMA interfaces, and the DAC output and ADC input are connected using radio frequency cables with BNC interfaces.

3. The FPGA-based underwater optical wireless real-time high-definition video transmission device according to claim 1 is characterized in that: The FPGA used is xc7z020clg400-2 series, the DAC used is AD9708, the ADC used is AD9238, the LED lamp is a blue / green light LED, the bias device used is ZFBT-4R2GW, and the detector used is avalanche photodiode Hamamatsu C12702 series.

Images