Perception and transmission integrated architecture, and wireless image transmission system and method based on perception and transmission integrated architecture

Abstract

The invention discloses a perception and transmission integrated architecture, and a wireless image transmission system and method based on the perception and transmission integrated architecture. The wireless image transmission system comprises a perception module and a transmission module, the sensing module comprises a Fourier lens, a micro lens array and a photoelectric detector array; the Fourier lens is used for carrying out Fourier transform on an image in a spatial domain into a frequency domain to obtain a frequency domain image, the microlens array is located on a rear focal plane of the Fourier lens and is used for separating phase information of the frequency domain image, and the photoelectric detector array is located on a rear focal plane of the microlens array and is used for collecting phase information and amplitude information of the frequency domain image; according to the invention, analog modulation of signals is realized while image perception is completed, one-time reading and one-time sending of all pixels are realized, and the whole process from analog signal acquisition to analog signal sending is completed in one step.

Description

Technical Field

[0001] The present invention discloses an integrated architecture for sensing and transmission, a wireless image transmission system and method based on the integrated architecture for sensing and transmission, belonging to the technical fields of semiconductor optoelectronic devices and optics. Background Art

[0002] In the era of the Internet of Things, especially the rapid development of the low-altitude economy has given rise to the development of a series of scientific and technological technologies. Among them, the wireless image sensing technology, which is one of the fastest and most important technologies, has received extensive attention. In the traditional wireless image communication architecture, the image sensing and information sending modules are physically separated. It is necessary to discretize the image into individual pixel points in space and then use a sensor array to collect signals; the collected photocurrent analog signal needs to be converted into a digital signal through an analog-to-digital converter; then, through a digital modulation module, the digital signal is modulated and then converted into an analog voltage signal and sent out through an antenna. The whole process needs to undergo two large amounts of signal form conversions of data (analog to digital, digital to analog), which is extremely energy-consuming and will bring a high signal delay.

[0003] For the wireless image transmission architecture, the existing methods generally introduce a compression module. After the photocurrent signal is collected at the sensor end and converted into a digital signal through analog-to-digital conversion, the original data is compressed by the data compression module, and finally the compressed data is digitally modulated by the modulation module and then converted into an analog voltage signal and sent out through an antenna. This process also faces multiple analog-to-digital conversion processes. The main reduction in energy consumption is due to the reduction in the amount of data that the modulation module needs to modulate. Although introducing a compression module can reduce the amount of data in the image transmission process to a certain extent, it consumes additional computing resources. Therefore, this solution has extremely limited improvement in system energy consumption and performance.

[0004] Especially for satellite remote sensing image acquisition, it is currently mainly restricted by two factors:

[0005] On the one hand, with the improvement of the resolution of remote sensing satellites and the increase in the observation frequency, the amount of generated data has increased sharply. Although on-orbit processing technology has been continuously improving, the on-orbit computing power and algorithm accuracy are still limited, which leads to low efficiency of on-orbit data preprocessing and intelligent interpretation, and complex data processing tasks need to be completed with ground support.

[0006] On the other hand, the data transmission bandwidth between the satellite and the ground station is limited, which restricts the data transmission speed and frequency. Especially when transmitting high-resolution images, the data volume is large and the transmission time is long, affecting the timeliness of the data. Secondly, the distance and communication time window between the satellite and the ground station are limited, which further restricts the data transmission efficiency. For example, the transit time of low-earth orbit satellites is short, and data transmission needs to be completed within a limited time. The bandwidth resources for satellite-to-ground data transmission are extremely limited.

[0007] For remote sensing image acquisition, the existing method is to carry high-performance computing chips on the satellite, preprocess and compress the remote sensing data before transmitting it to the ground receiving station. This process will inevitably cause extremely high energy consumption; another method is to establish a supercomputing center in space to realize the in-orbit processing of satellite data and send the operation results back to the ground receiving station. However, the construction of the supercomputing center requires a huge amount of human and material resources. In short, at present, how to more effectively realize the acquisition, data compression and transmission of ultra-wide area images by remote sensing satellites under the conditions of limited resources and energy is still a huge challenge.

[0008] In summary, there is an urgent need to develop a wireless image transmission technology that integrates image sensing, information compression and data modulation. Summary of the Invention

[0009] The object of the present invention: In order to overcome the contradiction between the ultra-large data volume and limited transmission bandwidth resources faced by the existing wireless image transmission system when collecting and transmitting ultra-wide area images, the present invention proposes a perception-transmission integrated architecture, a wireless image transmission system and method based on the perception-transmission integrated architecture, which realizes the analog modulation of signals while completing image sensing, and through frequency coding, realizes the one-time readout and one-time transmission of all pixels, and completes the whole process from analog signal acquisition to analog signal transmission in one step.

[0010] Technical solution: A perception-transmission integrated architecture, including: a perception module and a transmission module;

[0011] The perception module includes: a Fourier lens, a microlens array and a photodetector array; the Fourier lens is used to perform Fourier transform on the image in the spatial domain to the frequency domain to obtain a frequency-domain image, the microlens array is located on the rear focal plane of the Fourier lens and is used to separate the phase information of the frequency-domain image, and the photodetector array is located at the rear focal plane of the microlens array and is used to collect the phase information and amplitude information of the frequency-domain image;

[0012] Among them, the drain ends of each photodetector device in the photodetector array are independent, and the source ends are connected together; an AC voltage signal with the same amplitude but different frequencies is provided for each photodetector device as a driving signal; the frequency of each driving signal corresponds to the photodetector one by one; the mixed current signals output by each photodetector device are read out at the source end of the photodetector array; the output mixed current signals are subjected to Fourier transform to obtain frequency-domain signals of each frequency; by analyzing the relative intensities of the frequency-domain signals of each frequency, the phase information of the frequency-domain image is obtained, and by analyzing the absolute intensities of the frequency-domain signals of each frequency, the amplitude information of the frequency-domain image is obtained, realizing image perception;

[0013] The transmission module is used to send out the mixed current signal to realize one-time image sending.

[0014] Furthermore, the photodetector device uses a two-dimensional semiconductor material as the channel photosensitive material.

[0015] Furthermore, in the transmission module, the sending out of the mixed current signal to realize image sending specifically includes:

[0016] The mixed current signal is converted into a voltage signal, a carrier signal is superimposed on the voltage signal by using coherent modulation, and then it is sent out by using an antenna to realize image sending.

[0017] Furthermore, the frequency-domain image obtained by using a Fourier lens can be a frequency-domain image that only contains the main content of the image.

[0018] The present invention discloses a wireless image transmission system based on a perception and transmission integrated architecture, including:

[0019] An image sending end, which is used to send an image based on the above-disclosed perception and transmission integrated architecture;

[0020] An image receiving end, which is used to restore the original signal after receiving the signal through an antenna, and then obtain the frequency-domain information of the original image through analysis, and further obtain the spatial information of the original image through inverse Fourier transform.

[0021] The present invention discloses a wireless image transmission method based on a perception and transmission integrated architecture, including the following steps:

[0022] Build the above-disclosed perception and transmission integrated architecture;

[0023] Provide an AC voltage signal with the same amplitude but different frequencies for each photodetector device as a driving signal; the frequency of each driving signal corresponds to the photodetector one by one;

[0024] Read out the mixed current signals output by each photodetector device at the source end of the photodetector array;

[0025] Convert the output mixed current signal into a voltage signal;

[0026] Adopt coherent modulation to superimpose a carrier signal on this voltage signal and transmit it using an antenna to achieve image transmission.

[0027] Advantageous effects: Compared with the prior art, the present invention has the following advantages:

[0028] (1) In the existing wireless image sensing system, since the sensing module and the modulation module are physically separated, after the sensor end converts the image information into a photocurrent signal, it is converted into a digital signal through an analog-to-digital converter, and then after digital modulation by the modulation module, the signal is converted into an analog signal and transmitted through an antenna. The whole process needs to experience two large amounts of data signal form conversions (analog to digital, digital to analog), which has extremely high energy consumption and will bring relatively high signal delay. This is obviously disadvantageous for a wireless image transmission system that requires real-time performance and low power consumption. Especially in the face of the scenario of satellite remote sensing imaging, the huge demand for sending sensor data and the extremely limited downlink transmission bandwidth resources form an irreconcilable contradiction. To solve this problem to a certain extent, the integrated sensing and transmission architecture proposed by the present invention can achieve analog modulation of the signal while completing image sensing, and by adopting an innovative frequency coding method, it can achieve one-time reading and one-time transmission of all pixels, and complete the process from analog signal acquisition to analog signal transmission in one step; therefore, the integrated sensing and transmission architecture proposed by the present invention has outstanding effects in wireless sensing systems such as remote sensing image acquisition and high-altitude image acquisition by unmanned aerial vehicles;

[0029] (2) Most natural images are sparse, which means that the main information such as the contours of the images is concentrated in the low-frequency part in the frequency domain. To further reduce energy consumption, the present invention uses a pre-optical system to achieve compression of image information. By only collecting the low-frequency part of the signal, the basic restoration of the original image can be completed. Therefore, the present invention achieves great compression of image data through a pre-Fourier lens, with zero energy consumption generated. At the same time, the energy consumption required for transmitting the compressed data will also be further reduced. Description of the Drawings

[0030] Figure 1 It is a schematic diagram of wireless image transmission based on an integrated sensing and transmission architecture;

[0031] Figure 2 It is a performance display diagram of a wireless image transmission method based on an integrated sensing and transmission architecture proposed by the present invention;

[0032] Figure 3 It is a schematic diagram of the working mode of a single Fourier unit;

[0033] Figure 4 Schematic diagram of the hardware of the integrated sensing and transmission architecture;

[0034] Figure 5 Physical diagram of the hardware of the integrated sensing and transmission architecture. Specific implementation mode

[0035] The technical solution of the present invention will be further elaborated below in conjunction with the accompanying drawings and embodiments.

[0036] Embodiment 1:

[0037] This embodiment proposes an integrated sensing and transmission architecture based on the AC optical response of a two-dimensional semiconductor photodetector, which can be applied in a remote sensing satellite imaging and radio data transmission scenario as shown in (a) of Figure 1 . As shown in (b) of Figure 1 , this architecture includes a Fourier lens, a microlens array, and a two-dimensional semiconductor photodetector array.

[0038] The Fourier lens can perform a Fourier transform on the image in the spatial domain to the frequency domain, converting the discrete pixel spatial information into low-frequency or high-frequency information in the frequency domain; the microlens array is located on the rear focal plane of the Fourier lens, and the Shack-Hartmann principle is used to separate the phase information of the frequency-domain image; the two-dimensional semiconductor photodetector array is located at the rear focal plane of the microlens array and is used to collect the image frequency-domain information. The image frequency-domain information consists of two parts: amplitude information and phase information. The intensity of the light spot can represent the intensity information, and the phase information is related to the propagation direction of the light. The microlens array can be used to deflect the light rays with different phases to different positions below each microlens, that is, the light spot position encodes the phase information. There are four photodetection devices below each microlens. These four photodetection devices form a Fourier unit. The drain ends of the four photodetection devices are independent, and the source ends are connected together. The amplitude and phase of a frequency information in the frequency domain can be collected through frequency encoding.

[0039] The photodetection device in this embodiment uses a two-dimensional semiconductor material as the channel photosensitive material, benefiting from its ultra-fast carrier mobility, strong photoconductive gain, and a fabrication process compatible with CMOS chips. In this embodiment, molybdenum sulfide is used as the channel photosensitive material. Figure 1 As shown in (c) of

[0040] The specific operation of the frequency encoding adopted in this embodiment is as follows:

[0041] When the two-dimensional semiconductor photodetector array is working, the driving signal for each single photodetector is an AC voltage signal with the same amplitude (1V) but different frequencies (here, for example, f1 = 100Hz, f2 = 200Hz, f3 = 300Hz, f4 = 400Hz). Each frequency signal corresponds to a photodetector one by one, and the position of the photodetector is related to the phase. Therefore, the phase information is encoded with the frequency of the driving signal. When each photodetector is working, the position of the light spot will deflect to different photodetectors due to different phases (when the phase is 0, the light spot position is in the center);

[0042] At the source end, the mixed current signals output by the four photodetectors are read, and after Fourier transform, signals of four frequencies f1, f2, f3, and f4 can be obtained; Figure 1 (d) in shows that voltage signals of N frequencies are input into the microlens array, and the output is a mixed-frequency current signal containing image information. After Fourier transform, the corresponding current outputs under different frequency electrical signal inputs and different intensity optical signal inputs can be obtained.

[0043] By judging the relative intensities of the four frequency signals, the position of the light spot can be judged, and then the phase information of the frequency-domain image can be obtained. The amplitude of the strongest frequency signal represents the amplitude information of the frequency-domain image.

[0044] Due to the introduction of an additional frequency dimension, the outputs of all optoelectronic devices can be summed up by current summation, and the intensity information of each frequency signal can still be obtained when expanded in the frequency domain.

[0045] At the same time, for such a mixed current signal containing the complete spectral information of the image, a high-frequency carrier signal (the carrier frequency f = 118.3MHz used in this embodiment) can be superimposed by means of coherent modulation, and then the signal can be sent out using an antenna.

[0046] At the receiving end, after receiving the signal through the antenna, the original signal can be restored by means of coherent demodulation, and then the frequency-domain information of the original image can be obtained through simple software analysis, and then the spatial information of the original image can be obtained through inverse Fourier transform.

[0047] It should be noted that the method proposed in this embodiment has the following advantages:

[0048] 1. Since the frequency-domain information is collected, the compression of image information can be directly achieved by only collecting low-frequency information (including the main content of the image);

[0049] 2. By utilizing the AC optical response characteristics of the optoelectronic detection device, signal modulation can be completed while perceiving the image. All processes from image perception to signal transmission are carried out entirely in the analog domain without involving any analog-to-digital conversion process, reducing the energy consumption generated during the signal conversion process;

[0050] 3. By adopting the frequency encoding method, one-step readout of all pixels can be achieved, greatly reducing the latency.

[0051] Figure 2 The left picture shows the comparison of the original image transmission data volume (Original data) and the transmission data volume after this embodiment (Compressed data) in the experiment, and the data compression ratio is 30:1; the middle image represents the visualization effect of the original FashionMNIST dataset and the dataset reconstructed after simulation; Figure 2 The right picture on the right is the change in accuracy during the training process of the same network for the above two datasets. It can be seen that even though the images reconstructed using the compressed data (Reconstructed images) will lose some details, while greatly reducing the data transmission bandwidth, the information of the main body is retained, and a relatively high recognition accuracy can still be obtained during the neural network training process.

[0052] Embodiment 2:

[0053] Based on the concept of the integrated perception and transmission architecture proposed in Embodiment 1, a specific implementation scheme at the hardware level is provided at the hardware level now.

[0054] The preparation process of the two-dimensional semiconductor optoelectronic detection array used in this embodiment is as follows: First, the bottom electrode (Ti 2nm / Au 20nm) is prepared on the SiO2(280nm) / Si substrate through standard photolithography and electron beam evaporation (EBE) processes; 8nm of hafnium oxide is grown as the dielectric layer by atomic layer deposition (ALD); then the molybdenum sulfide thin film material grown by chemical vapor deposition (CVD) is transferred to the specified area; the top electrode (Sb 20nm / Au 20nm) is deposited by ultraviolet photolithography and electron beam evaporation; finally, the molybdenum sulfide thin film material is etched into the specified shape by inductively coupled plasma etching (ICP).

[0055] Connect the source ends of four optoelectronic detection devices together to form a Fourier unit, and apply AC voltage signals with different frequencies to the four optoelectronic detection devices. As Figure 3As shown in (a), light of different phases is deflected to different positions of the molybdenum sulfide device through a microlens array. For a beam of fixed phase, the microlens will deflect the beam to a specific direction. Therefore, the position of the photodetector is moved to control the light spot to illuminate different positions of the Fourier unit to simulate the deflection of the illumination position of light of different phases after passing through the microlens. Five different illumination positions are simulated, corresponding to five different phases: -pi (upper left corner), -pi / 2 (upper right corner), 0 (center position), pi / 2 (lower left corner) and pi (lower right corner).

[0056] Four photoelectric detectors in the Fourier unit are respectively applied with four different frequencies (f1 = 100 Hz, f2 = 200 Hz, f3 = 300 Hz, f4 = 400 Hz) and AC signals of the same amplitude (Amplitude = 1 V). At the same time, the output current signal I with different phases is read at the source end. light The experimental results are as follows Figure 3 As shown in (b) to (f), the left side is the output time domain current signal, and the right side is the frequency domain signal after Fourier transform. Figure 3 (b) shows the photocurrent output when the phase φ = -π, Figure 3 (c) shows the photocurrent output when the phase φ = -π / 2. Figure 3 (d) shows the photocurrent output when the phase φ = 0. Figure 3 (e) shows the photocurrent output when the phase φ = π / 2. Figure 3 (f) in FIG. 1 shows the photocurrent output when the phase φ=π.

[0057] It is worth noting that at different phases, that is, when the position of the light spot is different, both the output waveform and the spectrum information of the device are obviously different. Since we know the relationship between each input frequency electrical signal and the position a priori, we can determine the position of the light spot by analyzing the relative intensity of the output of different frequency components, and then deduce the phase information. The intensity information is obtained by analyzing the absolute intensity of the output of different frequency components.

[0058] At this point, the phase information and intensity information of the image are simultaneously encoded into the mixed frequency signal output by the photodetector through frequency encoding. By connecting multiple Fourier units together, complete perception of the image frequency domain information (including intensity and phase information) can be achieved at one time.

[0059] Figure 4 The process of building a complete transceiver system based on the integrated sensing architecture is demonstrated. Figure 4(a) in it shows a schematic architecture diagram of an image perception and wireless transceiver system. An array composed of multiple Fourier units jointly is used to collect the entire frequency-domain image information, and the final mixed output signal I of the array light contains the amplitude information and phase information of multiple frequencies in the image frequency domain. Then, through an operational amplifier module (SR570), the current signal I light is converted into a voltage signal V out . The voltage signal V out and a high-frequency carrier signal (fcarrier = 118.3 MHz, generated by Keysight 81150A signal generator) are superimposed through a multiplier (Mixer). Then, the superimposed signal is sent out through an antenna after passing through a power amplifier (Power AMP). After the signal is received by the antenna on the other side, first, it passes through a low-noise amplifier (LNA) and a low-pass filter (LPF), and then the original signal is demodulated from the received signal through coherent demodulation, and the signal is collected by an oscilloscope (Oscilloscope). This signal needs to pass through a low-pass filter to remove the high-frequency components in it and restore the original signal as much as possible. Then, the analog signal is converted into a digital signal through an analog-to-digital converter and processed to obtain compressed image data, thereby realizing the reconstruction of the original image. Figure 4 (b) in it shows the output of the photocurrent time-domain signal after the input image information is compressed and sensed by a molybdenum sulfide array. Figure 4 (c) in it shows the output signal after the baseband signal carrying image information is modulated by a 118.3 MHz carrier signal; Figure 4 (d) in it shows the time-domain signal containing image information obtained after coherent demodulation at the receiving end; Figure 4 (e) in it shows the reconstructed image obtained after the compressed image data is received and processed by the cloud.

[0060] Figure 5 On the left side is a microscope picture of the molybdenum sulfide array used in this embodiment (magnification is 50 times), and on the right side is a hardware physical picture of the complete transceiver system from image perception to signal transmission and reception built in this embodiment, including a laser output optical path (used to simulate the loading of image information), a molybdenum sulfide photosensitive array, a switch matrix, an AC signal source, a carrier signal generator, an antenna, an oscilloscope, etc. The screen shows the comparison between the transmitted image and the received image.

Claims

1. A perception and transmission integrated architecture, characterized by: include: Perception module and transmission module; The sensing module comprises: a Fourier lens, a microlens array and a photodetector array; the Fourier lens is used to perform Fourier transform of the image in the spatial domain into the frequency domain to obtain a frequency domain image; the microlens array is located on the rear focal plane of the Fourier lens and is used to separate the phase information of the frequency domain image; the photodetector array is located at the rear focal plane of the microlens array and is used to collect the phase information and amplitude information of the frequency domain image; The drain end of each photodetector device in the photodetector array is independent, and the source end is connected together; an AC voltage signal with the same amplitude but different frequencies is provided as a driving signal for each photodetector device; the frequency of each driving signal corresponds to the photodetector one by one; the mixed current signal output by each photodetector device is read out at the source end of the photodetector array; the output mixed current signal is Fourier transformed to obtain the frequency domain signal of each frequency; the phase information of the frequency domain image is obtained by analyzing the relative strength of the frequency domain signal of each frequency, and the amplitude information of the frequency domain image is obtained by analyzing the absolute strength of the frequency domain signal of each frequency, so as to realize image perception; The transmission module is used to send out the mixed current signal to achieve one-time transmission of the image.

2. The sensing and transmission integrated architecture according to claim 1, characterized in that: The photoelectric detection device uses a two-dimensional semiconductor material as a channel photosensitive material.

3. The perception transmission integrated architecture according to claim 1, characterized in that: In the transmission module, the mixed current signal is sent out to realize image transmission, which specifically includes: The mixed current signal is converted into a voltage signal, a carrier signal is superimposed on the voltage signal using coherent modulation, and then it is sent out using an antenna to achieve image transmission.

4. The perception transmission integrated architecture according to claim 1, characterized in that: The frequency domain image obtained by using the Fourier lens may be a frequency domain image containing only the main content of the image.

5. A wireless image transmission system based on a perception and transmission integrated architecture, characterized in that: include: An image sending end, used for sending images based on a perception transmission integrated architecture as described in any one of claims 1 to 4; The image receiving end is used to restore the original signal after receiving the signal through the antenna, and then obtain the frequency domain information of the original image through analysis, and then obtain the spatial information of the original image through inverse Fourier transform.

6. A wireless image transmission method based on a perception-transmission integrated architecture, characterized in that: The following steps are involved: Constructing a perception transmission integrated architecture as described in any one of claims 1 to 4; Providing an AC voltage signal with the same amplitude but different frequencies as a driving signal to each photoelectric detection device; the frequency of each driving signal corresponds to the photoelectric detector one by one; reading out the mixed current signal output by each photodetector device at the source end of the photodetector array; Convert the output mixed current signal into a voltage signal; A carrier signal is superimposed on the voltage signal using coherent modulation and sent out using an antenna to achieve image transmission.

Images