Color image sensor based on two-dimensional semiconductor heterostructure and its application in image convolution processing

Abstract

The application discloses a kind of color image sensor based on two-dimensional semiconductor heterostructure and its application in image convolution processing.The device contains deposited on substrate bipolar two-dimensional semiconductor and N-type two-dimensional semiconductor, which form horizontal heterojunction structure.By adjusting back gate voltage, the energy band arrangement of heterojunction is controlled, and a continuously variable spectral response function is realized, so that multi-band image information can be obtained without color filter.Meanwhile, by source-drain voltage, the photocurrent response of heterojunction is controlled, and then the output signal is given different weights.Several heterojunctions with the same size as the convolution kernel are connected in parallel, and the sum of the currents read is mapped to the gray value, and the convolved image under this waveband can be obtained.By applying convolution operation under different wavebands respectively, image convolution processing can be completed.The device performs well in integrated image acquisition and convolution calculation, effectively reduces the computing resource overhead, and significantly improves the image processing efficiency.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of new semiconductor devices, and relates to a color image sensor based on a two-dimensional semiconductor heterostructure and application of the color image sensor to image convolution processing. BACKGROUND

[0002] Image information usually contains multiple dimensions, such as light intensity, spatial position, color, etc. Collecting and processing high-dimensional image information helps to apply machine vision in life and production and improve work efficiency. Light intensity and spatial information are the most basic, and are usually obtained by using a CCD or CMOS array in combination with a readout circuit. However, with the expansion of application scenarios of machine vision, more complex application scenarios and tasks usually also rely on the collection and processing of color information.

[0003] In summary, there are still problems in the current machine vision system for realizing the recognition and processing of color images: the traditional CCD or CMOS needs to be combined with a complex structure of a color filter array, which increases the device size and process complexity; the spectral response curve of the traditional image sensor is fixed and cannot be modulated as needed; and the modules for processing and collecting information in the traditional color image recognition system are separated, which means that additional computing devices need to be introduced to complete the processing process of the color image, such as the commonly used convolution calculation, thereby increasing the computing resource overhead and processing time. SUMMARY

[0004] The purpose of the present application is to design a new type of color image sensor to solve the three problems of fixed sensing wavelength range, overly complex sensor structure and separation of sensing and algorithm implementation devices in the traditional machine vision system for processing color images.

[0005] The purpose of the present application can be achieved by the following technical solutions:

[0006] According to a first aspect of the present application, a color image sensor based on a two-dimensional semiconductor heterostructure is provided, which includes a bipolar two-dimensional semiconductor layer and an N-type two-dimensional semiconductor layer deposited on a substrate, and the two form a horizontal heterojunction. The color image sensor can realize the collection and processing of image data with convolution calculation at the same time through the combination of gate voltage and source-drain voltage.

[0007] Further, the substrate is a P-type heavily doped silicon wafer with deposited silicon oxide.

[0008] Further, the material of the bipolar two-dimensional semiconductor layer is molybdenum telluride.

[0009] Further, the material of the N-type two-dimensional semiconductor layer is molybdenum sulfide.

[0010] Further, by adjusting the gate voltage to control the band arrangement of the heterojunction, different spectral sensing functions are realized, and multi-channel image information is collected without color filters.

[0011] Further, by adjusting the source-drain voltage to control the photocurrent response intensity of the heterojunction, different waveband light signals are given different channel weights, and the convolution calculation process of the image is realized.

[0012] According to a second aspect of the present specification, an application of the above-mentioned color image sensor based on a two-dimensional semiconductor heterojunction structure in image convolution processing is provided, specifically comprising:

[0013] Determine the range of light signal intensity and the channel type, and determine the mapping relationship between the light signal intensity and the gray value.

[0014] According to the color of interest, determine the channel type, and determine the gate voltage of the device according to the channel type.

[0015] According to the purpose of image convolution processing, determine the convolution kernel corresponding to each channel, calculate the weight distribution of the convolution kernel under each waveband and normalize it.

[0016] Establish the mapping relationship between the photocurrent and the light intensity of the device under the specified channel, and the mapping relationship between the weight and the source-drain voltage.

[0017] Arrange a plurality of same devices according to the size and shape of the convolution kernel, and connect them in parallel as a photosensitive array.

[0018] According to the convolution kernel weight distribution and the mapping relationship, determine the source-drain voltage matrix of the photosensitive array.

[0019] When collecting an image, apply the pre-determined gate voltage and source-drain voltage matrix under different channels to the photosensitive array, and read the photocurrent and.

[0020] Convert the photocurrent and into a gray value, thereby obtaining the image after convolution processing under different channels, and fusing to obtain a color image.

[0021] Further, the mapping relationship between the light signal intensity and the gray value is specifically: pre-calibrate the dynamic range of the color image sensor to obtain the upper limit P max and the lower limit P min of the detectable light signal intensity; the expected image gray value range is denoted as G; the mapping relationship between the light signal intensity and the gray value is established according to the following formula: P = qG, where P is the difference between P max and P min , and q is a proportional coefficient.

[0022] Further, the determination of the gate voltage is specifically:

[0023] According to the color of interest, c channel types are determined, denoted as C1, C2, …, Cc i ,…,C c , i = 1, 2, …, c, where C i represents the wavelength range corresponding to channel i;

[0024] Under the condition of using C1~Cc c wavelength range light signal to irradiate the device, the transfer curve of the device is tested, and the corresponding gate voltage V g1 , V g2 , …, V gi , …, V gc , i = 1, 2, …, c, are extracted, where V gi means that under this gate voltage, the responsivity of the device to C i is the highest.

[0025] Further, for channel C i , the gate voltage V gi is applied to the device, and the device is irradiated with C i wavelength range light signal, and the source-drain voltage V ds is scanned to test the output curve of the device, and the output curve of the photocurrent I ph is extracted, and the selection of V ds is based on the ability of the device to exhibit complete rectification characteristics, which is set to be (V ds，min , V ds，max ), and the C max wavelength range light signal intensity is adjusted within P min and P i to obtain multiple sets of photocurrent I ph output curves, which are fitted using the following formula:

[0026] I ph = α(kP) β

[0027] where P is the light signal intensity, α is the proportionality coefficient, k is a function of V ds , which is positively correlated with V ds , and its variation range represents the ability of V ds to adjust the output signal when the light intensity is fixed, and β reflects the linear relationship between the photocurrent and the input light intensity, and α and β are fixed values under C i wavelength range light signal, and this step is repeated to obtain the corresponding α and β values of each C i ;

[0028] where V ds，max corresponds to the maximum k value, denoted as k max , and V ds，min corresponds to the minimum k value, denoted as k min, divide all k values by k max to complete normalization, take the normalized k value as weight W, and fit the relationship between W and V under each C i according to the following formula: ds

[0029]

[0030] where a and b reflect the speed of change of W with V ds , and a and b are fixed values under C i band light signals, repeat this step to obtain the corresponding a and b values of each C i .

[0031] Further, the same device is arranged according to the pixel arrangement of the convolution kernel to form a photosensitive array, each pixel corresponds to a device, all devices are connected in parallel, and the weight of each pixel in the convolution kernel K i corresponding to channel C i is normalized, and then substituted into the fitting relationship formula of W and V i corresponding to channel C ds to obtain the source-drain voltage required to be applied to the device at the corresponding position of the pixel, thereby obtaining the source-drain voltage V i matrix under each C ds , denoted as V ds i .

[0032] Further, in order to perform convolution processing on the initial image to obtain a feature map, first, for each C c under the selected C1~C i wavelength range, apply V gi to the gate of each device in the photosensitive array, and the source-drain voltage is applied according to the matrix V ds i , read the current sum of all parallel devices, then maintain V gi unchanged, slide the convolution kernel, traverse the entire initial image, read the current sum multiple times, and convert the read current sum to a plurality of gray values G corresponding to C i , and the obtained gray value distribution map is the feature map after convolution processing under C i band, and combining all feature maps under C1~C c wavelength range to obtain the final multi-band color image after convolution processing.

[0033] ​The beneficial effects of the present application: compared with the pixel units used by the existing image sensor array, the present application designs a three-terminal device, which uses a bipolar two-dimensional semiconductor molybdenum telluride and an N-type two-dimensional semiconductor molybdenum sulfide to build a horizontal heterojunction on a heavily doped silicon wafer with deposited silicon oxide. Due to the small thickness of the material, it is easy to be regulated by the back gate. By applying voltage to the back gate to modulate the energy band arrangement of the heterojunction, the responsivity to different wavelength optical signals can be obtained, so that different waveband optical signals can be differentiated without using color filters. At the same time, adjusting the source-drain voltage of each device can adjust the readout photocurrent size, thereby scaling the photocurrent within a certain range. By connecting a number of sensing units with the same size of convolution kernel in parallel, the multiplication and accumulation operation of the convolution kernel and the corresponding pixel can be completed, which makes it possible to directly realize convolution processing under multiple wavebands at the device end. BRIEF DESCRIPTION OF DRAWINGS

[0034] Figure 1 Structure diagram of the heterojunction device built by the bipolar and N-type two-dimensional semiconductor;

[0035] Figure 2 Image convolution processing flowchart using the heterojunction device;

[0036] Figure 3 Normalized responsivity distribution of the device corresponding to 450-700nm optical signals when the gate voltage changes in the range of-50V-50V;

[0037] Figure 4 Sensing effect of the example blue letter "I" and red letter "C" under the corresponding gate voltage;

[0038] Figure 5 V ds Matrix distribution of the blue channel and the red channel calculated according to the convolution kernel;

[0039] Figure 6 Effect of using the heterojunction device to process a 64x64 color image containing Gaussian noise. DETAILED DESCRIPTION

[0040] The following are specific embodiments of the present application, which further illustrate the present application. It should be understood that these embodiments are only used to illustrate the present application and not to limit the scope of the present application. In addition, it should be understood that after reading the content taught by the present application, those skilled in the art can make various modifications or changes to the present application, and these equivalent forms fall within the scope defined by the claims attached hereto.

[0041] As Figure 1As shown, a heterojunction of bipolar two-dimensional semiconductor molybdenum telluride (MoTe2) under and N-type two-dimensional semiconductor molybdenum sulfide (MoS2) over is prepared on a P-type heavily doped silicon wafer with deposited silicon oxide as a color image sensor, which can realize the acquisition and processing of image data with convolution calculation at the same time through the combination of back gate voltage and source-drain voltage. Figure 2 As shown, the application of the color image sensor in image convolution processing is described in detail below.

[0042] Further, the dynamic range of the color image sensor, i.e. the upper and lower limits of the light signal intensity that can be detected, is calibrated in advance through experiments, the upper limit is denoted as P max , and the lower limit is denoted as P min ; the expected image gray value range is denoted as G; the mapping relationship between light signal intensity and gray value range is established according to formula (1):

[0043] P=qG (1)

[0044] wherein P is the difference between P max and P min , and q is a proportional coefficient.

[0045] Further, c channel types are determined according to the color of interest, denoted as C1, C2, …, C i , …, C c , i=1, 2, …, c, wherein C i represents the wave band corresponding to channel i, for example, if blue is needed to be concerned, C i in the range of 450nm-520nm can be selected.

[0046] Further, the transfer curve of the device is tested under the condition of using C1-C c wavelength range light signal irradiation device, and the corresponding gate voltages V g1 , V g2 , …, V gi , …, V gc are extracted, i=1, 2, …, c, wherein V gi means that the responsivity of the device to C i is the highest at this gate voltage.

[0047] Further, the corresponding c convolution kernels K iThe convolution kernels, i = 1, 2, ..., c, are determined based on the purpose of the image convolution processing. For example, a Sobel kernel is used to extract target edges; a Laplacian kernel is used for image sharpening; and a Gaussian kernel is used for Gaussian filtering. A Gaussian kernel is a two-dimensional square matrix of size m × m, where m is an odd number, typically 3. The standard deviation σ of the kernel is then determined based on the desired processing amplitude, for example, 2. After determining m and σ, the weight W at each position in the convolution kernel for each band can be determined. xy i , x=1,2,…,m, y=1,2,…,m, i=1,2,…,c.

[0048] Furthermore, for channel C i Apply gate voltage V to the device gi and using C i The device is illuminated by a light signal in the band, and the source-drain voltage V is scanned. ds To test the output curve of the device and extract the photocurrent I ph The output curve, V ds The selection criterion is to enable the device to exhibit complete rectification characteristics, assuming its range is (V ds，min V ds，max ), in P max With P min Internal adjustment C i The intensity of the optical signal in the band, i.e., the intensity P1, P2, ..., P j ,…,P n C, j = 1, 2, ..., n i n sets of photocurrents I were obtained from the band optical signal test. pj For the output curve, n needs to be chosen as a suitable value, at least greater than 4. The larger the value, the better it is for improving the accuracy of the subsequent fitting process.

[0049] Furthermore, for C of different intensities i n sets of photocurrents I obtained from the band optical signal pj The output curve is fitted using formula (2):

[0050] I ph =α(kP) β (2)

[0051] Where P is the optical signal intensity, α is the scaling factor, and k is the value of V. ds The function that is related to V ds They are positively correlated, and their range of variation represents V. ds The ability to adjust the output signal when the light intensity (optical power density) is constant, while β reflects the linear relationship between the photocurrent and the input light intensity, at a certain C iUnder optical signals, α and β are fixed values, meaning they are independent of light intensity and only related to the wavelength. Repeating this step yields each C. i The corresponding α and β values.

[0052] Furthermore, V ds，max The corresponding k value is the largest, which is k. max , and V ds，min The corresponding minimum value of k is k. min Divide all k values ​​by k max To complete the normalization, the normalized k value is regarded as the weight W, and each C is obtained by fitting according to formula (3). i W and V below ds Relationship:

[0053]

[0054] Among them, a and b reflect W as V increases. ds The rate of change, at a certain C i Under the optical signal, a and b are fixed values. Repeating this step yields each C. i The corresponding values ​​of a and b.

[0055] Furthermore, the same devices are arranged according to convolution kernel K i The pixels are arranged in a way that forms a photosensitive array, with each pixel corresponding to one device. All devices are connected in parallel, and the K-type sensors used are... i The weight W of each pixel in xy i After normalization, substituting the values ​​into formula (3) yields the source-drain voltage V that needs to be applied to the device at the corresponding pixel location. ds，xy This way we can get each C i V of size m×m ds A matrix, denoted as V ds i .

[0056] Furthermore, in practical applications, in order to perform convolution processing on the initial image to obtain feature maps, firstly, for the selected C1 to C2... c Each C in the wavelength range i Apply V to the gate of each device in the photosensitive array gi The source-drain voltage is determined based on the previously defined K. i The corresponding matrix V ds i To apply, read the sum of the currents of m×m parallel devices, and then maintain V gi Without changing the convolution kernel, iterate through the entire initial image, read the sum of currents multiple times, substitute the sum of the read currents into formula (2) to obtain the light intensity P, and then obtain C according to formula (1). iThe corresponding gray value G at this time is the gray value distribution diagram in C i The feature map after convolution processing in this wave band, combined with C1~C c All feature maps in the wavelength range can obtain the final multi-wave band color image after convolution processing.

[0057] Embodiment 1

[0058] The structure of the heterojunction device based on the above-mentioned bipolar and N-type two-dimensional semiconductor in this embodiment is shown in Figure 1 The substrate is a P-type heavily doped silicon wafer with a 285 nm thick silicon oxide deposited (resistivity 0.001-0.005 Ω·cm). The bipolar MoTe2 and N-type MoS2 films come from bulk materials that are mechanically exfoliated and stacked on the silicon oxide in turn by dry transfer technology. The window area of MoTe2 in contact with the metal electrode is subjected to an oxygen plasma cleaning process before deposition, with a power of 60 W and a flow rate of 50 sccm for 1 min. After that, the source and drain regions are determined by patterning, and a 40 nm thick gold film is deposited by thermal evaporation as the contact electrode. The P-type heavily doped silicon serves as the back gate. This device can simultaneously realize the perception and convolution processing of multi-wave band optical signals at the device level.

[0059] Embodiment 2

[0060] In this embodiment, the heterojunction device based on the above-mentioned bipolar and N-type two-dimensional semiconductor is irradiated with 450-700 nm laser light, and the photocurrent is tested under the conditions of a source-drain voltage of 2 V and a gate scanning range of -50 V to 50 V. Then, the responsivity is calculated according to the light intensity of different wave bands and normalized. Figure 3 The normalized responsivity distribution is shown. As the gate voltage increases, the peak responsivity corresponding to the wavelength gradually transitions from 450 nm to the vicinity of 750 nm, realizing the reconstruction of the spectral response function of the device under different gate voltages.

[0061] Embodiment 3

[0062] In this embodiment, based on the results of the above-mentioned embodiment 2, the gate voltage is selected as -50 V and the source-drain voltage is selected as 2 V to mainly perceive the 450 nm laser light, i.e., the blue channel; the gate voltage is selected as 0 V and the source-drain voltage is selected as 2 V to mainly perceive the 700 nm laser light, i.e., the red channel. Figure 4 The perception of the example blue letter "I" and the red letter "C" is demonstrated respectively. By comparing the intensity distribution of the initial image and the imaging effect under the two gate voltages, it can be seen that when different gate voltages are used, the optical signal of the corresponding main response wave band becomes more obvious, while the optical signal of the non-main response wave band becomes weaker.

[0063] Example 4

[0064] This embodiment demonstrates the use of V-type heterojunction devices based on the aforementioned bipolar and N-type two-dimensional semiconductors. ds The process of performing convolution calculations using matrices. For example... Figure 5 As shown, a 3×3 Gaussian convolution kernel with a standard deviation of 2 was selected, and the weights for each pixel were calculated after normalization. Following this... Figure 2 The process shown yields the coefficients in formula (3). Substituting the weights into formula (3) yields the V values ​​corresponding to the blue and red channels respectively. ds matrix.

[0065] Example 5

[0066] In this embodiment, the V corresponding to the blue and red channels obtained in Example 4 is used. ds The matrix represents the source-drain voltages of the heterojunction devices at the corresponding locations according to V. ds Matrix application. For example... Figure 6 As shown, taking a 64×64 color image containing Gaussian noise as an example, using a gate voltage of -50V, the V of the blue channel corresponding to the Gaussian convolution kernel... ds A matrix is ​​used to acquire the blue channel image; a Gaussian convolution kernel with a gate voltage of 0V is used for the red channel. ds A matrix is ​​used to acquire the red channel image. The initial color image is split into blue and red channels, acquired, and then subjected to Gaussian convolution, resulting in a significantly reduced noise in the output image.

[0067] The specific embodiments described herein are merely illustrative of the spirit of the invention. Those skilled in the art to which this invention pertains may make various modifications or additions to the described specific embodiments or use similar methods to substitute them, without departing from the spirit of the invention or exceeding the scope defined by the appended claims.

Claims

1. Use of a two-dimensional semiconductor heterostructure based color image sensor in image convolution processing, characterized in that, The color image sensor comprises a bipolar two-dimensional semiconductor layer and an N-type two-dimensional semiconductor layer deposited on a substrate, both forming a horizontal heterojunction, and is capable of simultaneously realizing the acquisition and processing of image data with convolution calculation through the combination of gate voltage and source-drain voltage. The range of light signal intensity and the channel type are determined, and the mapping relationship between the light signal intensity and the gray value is determined. The channel type is determined according to the color of interest, and the gate voltage of the device is determined according to the channel type. The convolution kernel corresponding to each channel is determined according to the purpose of image convolution processing, and the weight distribution of the convolution kernel under each waveband is calculated and normalized. The mapping relationship between the photoelectric current and the light intensity of the device under the specified channel, and the mapping relationship between the weight and the source-drain voltage are established. A plurality of same devices are arranged according to the size and shape of the convolution kernel and are connected in parallel as a photosensitive array. The source-drain voltage matrix of the photosensitive array is determined according to the convolution kernel weight distribution and the mapping relationship. When collecting an image, the gate voltage and the source-drain voltage matrix under the predetermined different channels are applied to the photosensitive array, and the photocurrent and are read. The photocurrent and are converted into gray values, thereby obtaining the images after convolution processing under different channels, and the color image is obtained by fusion.

2. Use according to claim 1, characterized in that, The mapping relationship between the light signal intensity and the gray value is specifically: a dynamic range of the color image sensor is calibrated in advance to obtain an upper limit P max and a lower limit P min of the detectable light signal intensity; The range of the image gray value to be obtained is denoted as G; the mapping relationship between the light signal intensity and the gray value is established according to the following formula: P=qG, wherein P is the P max The difference between P min and P 3. Use according to claim 2, characterized in that, The determination of the gate voltage is specifically as follows: According to the color to be concerned, c channel categories are determined, denoted as C1, C2, …, C i ,…,C c , i = 1, 2, …, c, wherein C i represents the wave band corresponding to channel i; In the case of using C1-C c The transfer curve of the device is tested under the condition of the wavelength range light signal irradiation device, and the corresponding gate voltage V g1 , V g2 ,…,V gi ,…,V gc , i = 1, 2, …, c, wherein V gi means that the responsivity of the device to C i is the highest under the gate voltage.

4. Use according to claim 3, characterized in that, For channel C i A gate voltage V gi is applied to the device, and C i band light signal is used to irradiate the device, scanning source-drain voltage V ds to test the output curve of the device, and the output curve of the photocurrent I ph is extracted, V ds The selection basis is that the device can exhibit complete rectification characteristics, and its range is (V ds，min , V ds，max ), and C max band light signal intensity is adjusted within P min and P i , to obtain multiple sets of output curves of photocurrent I ph , and the following formula is used for fitting: I ph = a(kP) β where P is the light signal intensity, a is a proportional coefficient, k is a function of V ds , which is positively related to V ds , and its variation range represents the variation range of V ds , the ability to adjust the output signal when the light intensity is fixed, and β reflects the linear relationship between the photocurrent and the input light intensity, and a and β are fixed values under the C i band light signal, and repeating this step operation obtains the corresponding a and β values of each C i ​ Where, V ds，max The maximum k value is k max And V ds，min The minimum k value is k min Divide all k values by k max To complete the normalization, the normalized k value is considered as the weight W, and the relationship between W and V under each C i is fitted as follows: ds ​ where a and b reflect the speed of change of W with V ds at C i band optical signals, and a and b are fixed values. This procedure is repeated to obtain the corresponding a and b values for each C i band optical signal.

5. Use according to claim 4, characterized in that, The same device is arranged according to the pixel arrangement mode of the convolution kernel to form a photosensitive array, each pixel corresponds to a device, all devices are connected in parallel, and the channel C i The weight of each pixel in the corresponding convolution kernel K i is normalized, and then substituted into the fitted channel C i The relationship formula of corresponding W and V ds is substituted into the relationship formula of corresponding W and V i , so that the source-drain voltage required to be applied to the device corresponding to the position of each pixel is obtained, and thus the source-drain voltage V ds under each C ds is obtained. i 6. Use according to claim 5, characterized in that, In order to obtain the feature map by convolution processing on the initial image, first, for the selected C1~C c Each C i , the gate of each device in the photosensitive array is applied with V gi , the source-drain voltage is applied according to the matrix V ds i , the current sum of all parallel devices is read, and then V gi is kept unchanged, the convolution kernel is slid, the entire initial image is traversed, the current sum is read multiple times, and the read current sums are converted into a plurality of gray scale values G corresponding to C i , and the obtained gray scale value distribution map is the feature map obtained by convolution processing in the C i band, and the final multi-band color image obtained by convolution processing is obtained by combining all the feature maps in the C1~C c wavelength range.

7. Use according to claim 1, characterized in that, The substrate is a P-type heavily doped silicon wafer with deposited silicon oxide; the material of the bipolar two-dimensional semiconductor layer is molybdenum telluride; and the material of the N-type two-dimensional semiconductor layer is molybdenum sulfide.

8. The use according to claim 1, characterized in that, The energy band arrangement of the heterojunction is controlled by adjusting the gate voltage, thereby realizing different spectral sensing functions and collecting multi-channel image information without color filters.

9. The use according to claim 1, characterized in that, The photocurrent response intensity of the heterojunction is controlled by adjusting the source-drain voltage, thereby giving different channel weights to different waveband light signals and realizing the convolution calculation process of the image.

Images