Integrated sensing and machine learning processing device

Abstract

The present disclosure provides a semiconductor device with integrated sensing and processing capabilities. The semiconductor device includes a sensing module configured to generate a plurality of analog sensing signals and a machine learning (ML) processor. The sensing module and the ML processor are fabricated on a single wafer. The ML processor includes a crossbar array that processes the analog sensing signals to generate analog pre-processed sensing data, an analog-to-digital converter (ADC) that converts the analog pre-processed sensing data into digital pre-processed sensing data, and a machine learning processing unit that processes the digital pre-processed sensing data using one or more machine learning models. [Representative image] Figure 4B

Description

[Technical Field]

[0001] TECHNICAL FIELD Embodiments of the present disclosure relate generally to computing devices, and more particularly to integrated sensing and machine learning processing devices. [Background technology]

[0002] Machine learning (ML) is widely used in facial recognition, speech recognition, natural language processing, image processing, and more. ML typically involves analyzing large amounts of sensing data based on complex machine learning models. Traditional edge devices (local devices in close proximity to sensors that collect sensing data) lack the computational power to perform such analyses. As a result, the sensing data generated by the sensors may need to be digitized and transmitted to a remote computing device (e.g., a data center) with ML processing capabilities. This involves digitizing large amounts of data and may require advanced communication capabilities, as well as significant energy and time consumption, to transfer the digitized sensing data. Transferring raw data from sensors to remote devices may raise privacy concerns, and encrypting the raw sensing data for secure data transfer may further increase the computational costs required for ML. Furthermore, some applications (e.g., medical applications that utilize ML) may require real-time data processing. Therefore, it may be desirable to run machine learning models locally on edge devices. However, traditional edge devices lack the integrated sensing and processing capabilities to locally extract information and features from analog sensing data provided by local sensors and perform ML processing. Summary of the Invention

[0003] The following is a simplified summary of the disclosure to provide a basic understanding of some aspects of the disclosure. This summary is not an extensive overview of the disclosure. It is not intended to identify key features or critical elements of the disclosure or to define the scope of the claims or particular embodiments of the disclosure. Its purpose is to present some concepts of the disclosure in a simplified form as a prelude to the more detailed description that is presented later.

[0004] According to one or more aspects of the present disclosure, a semiconductor device capable of functioning as an integrated sensing and machine learning processing device is provided. The semiconductor device may include a sensing module configured to generate a plurality of analog sensing signals and a machine learning (ML) processor. The ML processor may include one or more crossbar arrays configured to process the analog sensing signals to generate analog pre-processed sensing data, an analog-to-digital converter (ADC) configured to convert the analog pre-processed sensing data into digital pre-processed sensing data, and a machine learning processing unit configured to process the digital pre-processed sensing data using one or more machine learning models. The sensing module and the ML processor are fabricated on a single wafer.

[0005] In some embodiments, the sensing module and the ML processor are fabricated on the same side of the wafer.

[0006] In some embodiments, the sensing module is fabricated in a first portion of a wafer and the ML processor is fabricated in a second portion of the wafer surrounding the first portion of the wafer.

[0007] In some embodiments, the transistors of the ML processor and the sensors of the sensing module are fabricated on a substrate of a wafer, and the semiconductor device further includes a plurality of interconnect layers fabricated over the transistors and the plurality of photodiodes.

[0008] In some embodiments, the first metal via, the second metal via, and the third metal via of the first interconnect layer are connected to a source region, a gate region, and a drain region, respectively, of a first transistor of the plurality of transistors, and in some embodiments, the fourth metal via and the fifth metal via of the first interconnect layer are connected to a first photodiode of the plurality of photodiodes.

[0009] In some embodiments, the RRAM devices of the ML processor are fabricated on metal pads or metal vias of an interconnect layer, hi some embodiments, the RRAM devices are connected to the first transistors through the first interconnect layer.

[0010] In some embodiments, the sensing module includes a set of microlenses and a set of color filters, the microlenses and color filters of the sensing module being fabricated on multiple interconnect layers.

[0011] In some embodiments, the sensing module is fabricated on an ML processor.

[0012] In some embodiments, the sensing module and the ML processor are fabricated on different sides of a wafer.

[0013] In some embodiments, the sensing module includes an image sensor array, and the plurality of analog sensing signals includes a plurality of analog image signals.

[0014] In some embodiments, the analog pre-processed sensing data corresponds to a plurality of features extracted from the analog sensing signal, and the machine learning processing unit performs machine learning using the extracted features.

[0015] In some embodiments, the semiconductor device further includes a packaging substrate to which the wafer is connected via an interconnect layer.

[0016] In some embodiments, the ML processor is powered using the analog sensing signal.

[0017] In some embodiments, the semiconductor device further includes a transceiver configured to transmit predicted outputs generated by the machine learning processing unit based on the one or more machine learning models to a computing device, and to receive instructions from the computing device to perform operations based on the predicted outputs.

[0018] In some embodiments, the analog pre-processed sensing data represents a convolution of the analog sensing signal with a kernel.

[0019] In some embodiments, the conductance values ​​of multiple crosspoint devices in one or more crossbar arrays are programmed to values ​​representing the kernel.

[0020] In some embodiments, the sensing module includes a two-dimensional sensor array, and a plurality of crosspoint devices of the one or more crossbar arrays are configured to receive as inputs analog sensing signals generated by the two-dimensional sensor array.

[0021] In some embodiments, the one or more crossbar arrays include multiple crossbar arrays arranged in multiple different planes.

[0022] According to one or more aspects of the present disclosure, a semiconductor device includes a sensing module configured to generate a plurality of analog sensing signals and a machine learning processor configured to generate predicted outputs by processing the analog sensing signals using one or more machine learning models, the machine learning processor including a plurality of crossbar arrays configured to generate a plurality of analog outputs representative of the predicted outputs and an analog-to-digital conversion unit configured to convert the plurality of analog outputs representative of the predicted outputs into a plurality of digital signals representative of the predicted outputs, wherein the sensing module and the machine learning processor are fabricated on a wafer.

[0023] In some embodiments, the semiconductor device further includes a packaging substrate to which the wafer is connected via an interconnect layer. [Brief explanation of the drawings]

[0024] The present disclosure will be more fully understood from the following detailed description of various embodiments thereof and the accompanying drawings, which are not intended to limit the disclosure to particular embodiments but are intended for purposes of illustration and understanding.

[0025] [Figure 1A] FIG. 1A is a schematic diagram illustrating an example of a processing device with integrated sensing and processing capabilities, according to some embodiments of the present disclosure. [Figure 1B] FIG. 1B is a schematic diagram illustrating an example of a processing device with integrated sensing and processing capabilities, according to some embodiments of the present disclosure. [Figure 2] FIG. 2 is a diagram illustrating an example of a crossbar array according to some embodiments of the present disclosure. [Figure 3] FIG. 3 is a schematic diagram illustrating an example of a three-dimensional crossbar array, according to some embodiments of the present disclosure. [Figure 4A] FIG. 4A is a schematic diagram illustrating an example of a semiconductor device that can function as a machine learning processor, according to some embodiments of the present disclosure. [Figure 4B]FIG. 4B is a schematic diagram illustrating an example of a semiconductor device that can function as a machine learning processor, according to some embodiments of the present disclosure. [Figure 5A] FIG. 5A illustrates a cross-sectional view of an example image sensor wafer, according to some embodiments of the present disclosure. [Figure 5B] FIG. 5B illustrates a cross-sectional view of an example image sensor wafer, according to some embodiments of the present disclosure. [Figure 6A] FIG. 6A is a schematic diagram illustrating a cross-sectional view of an example CMOS circuit including a CMOS-compatible RRAM, according to some embodiments of the present disclosure. [Figure 6B] FIG. 6B is a schematic diagram illustrating a cross-sectional view of an example CMOS circuit including a CMOS-compatible RRAM, according to some embodiments of the present disclosure. [Figure 7A] FIG. 7A is a schematic diagram illustrating a cross-sectional view and a top view, respectively, of a monolithic wafer with integrated sensing and ML processing capabilities, according to some embodiments of the present disclosure. [Figure 7B] FIG. 7B is a schematic diagram illustrating a cross-sectional view and a top view, respectively, of a monolithic wafer with integrated sensing and ML processing capabilities, according to some embodiments of the present disclosure. [Figure 7C] FIG. 7C is a schematic diagram showing a cross-sectional view of a structure for fabricating a monolithic wafer as described in connection with FIGS. 7A-7B. [Figure 7D] FIG. 7D is a schematic diagram showing a cross-sectional view of a structure for fabricating a monolithic wafer as described in connection with FIGS. 7A-7B. [Figure 7E] FIG. 7E is a schematic diagram showing a cross-sectional view of a structure for fabricating a monolithic wafer as described in connection with FIGS. 7A-7B. [Figure 7F] FIG. 7F is a schematic diagram showing a cross-sectional view of the structure for fabricating the monolithic wafer described in connection with FIGS. 7A-7B. [Figure 7G] FIG. 7G is a schematic diagram showing a cross-sectional view of a structure for fabricating a monolithic wafer as described in connection with FIGS. 7A-7B. [Figure 7H] FIG. 7H is a schematic diagram showing a cross-sectional view of a structure for fabricating a monolithic wafer as described in connection with FIGS. 7A-7B. [Figure 7I] FIG. 7I is a schematic diagram showing a cross-sectional view of a structure for fabricating a monolithic wafer as described in connection with FIGS. 7A-7B. [Figure 8A] FIG. 8A is a schematic diagram illustrating a cross-sectional view of an example monolithic wafer with integrated sensing and processing capabilities, according to some embodiments of the present disclosure. [Figure 8B] FIG. 8B is a schematic diagram illustrating a cross-sectional view of an example monolithic wafer with integrated sensing and processing capabilities, according to some embodiments of the present disclosure. [Figure 8C] FIG. 8C illustrates a cross-sectional view of an example structure for fabricating a monolithic wafer as described in connection with FIGS. 8A-8B, according to some embodiments of the present disclosure. [Figure 8D] FIG. 8D illustrates a cross-sectional view of an example structure for fabricating a monolithic wafer as described in connection with FIGS. 8A-8B, according to some embodiments of the present disclosure. [Figure 8E] FIG. 8E illustrates a cross-sectional view of an example structure for fabricating a monolithic wafer as described in connection with FIGS. 8A-8B, according to some embodiments of the present disclosure. [Figure 8F] FIG. 8F illustrates a cross-sectional view of an example structure for fabricating a monolithic wafer as described in connection with FIGS. 8A-8B, according to some embodiments of the present disclosure. [Figure 9A] FIG. 9A is a schematic diagram illustrating a cross-sectional view of an example monolithic wafer with integrated sensing and processing capabilities, according to some embodiments of the present disclosure. [Figure 9B] FIG. 9B is a schematic diagram illustrating an example of the monolithic wafer shown in FIG. 9A. DETAILED DESCRIPTION OF THE INVENTION

[0026] Aspects of the present disclosure provide a processing device with integrated sensing and machine learning capabilities and a method for manufacturing the same. The processing device according to the present disclosure can include a sensing module and a machine learning (ML) processor integrated on the same semiconductor device using three-dimensional (3D) chiplet integration or monolithic 3D integration. The sensing module can include a sensor array that generates analog sensing data (e.g., analog image signals generated by an image sensor). The ML processor can process the analog sensing data using one or more machine learning models.

[0027] In one implementation, the ML processor may include a preprocessing unit capable of preprocessing analog sensing data for ML processing by feature extraction, dimensionality reduction, image processing, etc. The preprocessing unit may include one or more crossbar arrays capable of preprocessing the analog sensing data in the analog domain. Each crossbar array may be a circuit structure having interconnected electrically conductive lines sandwiching a resistive switching material at their crosspoints. The resistive switching material may include, for example, memristors (also known as resistive random access memory (RRAM) or ReRAM)). The analog sensing data may be provided as an input signal to the crossbar array. The crossbar array may generate an analog output signal representing the preprocessed sensing data. The analog output signal is then converted to a digital signal representing the preprocessed sensing data for subsequent machine learning processing by the ML processor. By preprocessing the analog sensing data in the analog domain and digitizing the preprocessed sensing data rather than the raw sensing data, the ML processor described herein enables significant data reduction because only a small amount of information (e.g., the preprocessed sensing data) is digitized and transmitted from the edge sensing module to the next layer of the network.

[0028] In another implementation, the ML processor can run a machine learning model on the analog sensing data and generate analog signals representing the predicted output of the ML processing (e.g., classification results, labels assigned to the analog sensing data, outputs of layers of a neural network, decisions made based on the ML model, etc.). For example, the ML processor can utilize a crossbar array to implement a multi-layer neural network. The analog output signal generated by the crossbar array represents the predicted output and can be converted to a digital output and transmitted to another computing device.

[0029] In some embodiments, the processing device can be implemented using monolithic wafer integration. For example, the sensing module can be monolithically integrated with circuitry implementing the ML processor on a single substrate (e.g., a single crystal silicon substrate). The ML processor can include one or more CMOS circuits including transistors, RRAM devices, interconnect layers, and / or any other suitable components for implementing the ML processor. The sensing module can include photodiodes, interconnect layers, color filters, microlenses, and / or any other suitable components for implementing the sensing module.

[0030] In one implementation, the sensing module and the ML processor can be fabricated in non-overlapping portions of the substrate. For example, the sensing module can be fabricated in a central portion of the substrate. The ML processor can be fabricated in a second portion of the substrate surrounding the central portion of the substrate. The sensing module and the ML processor can be fabricated on the same side of the substrate. In another implementation, the sensing module and the ML processor can be fabricated on different sides of the substrate. For example, CMOS circuitry implementing the ML processor can be fabricated on a first side of the substrate. The substrate can then be flipped over to expose a second side of the substrate (e.g., the backside of the substrate). The sensing module can be fabricated on the second side of the substrate. In yet another implementation, CMOS circuitry implementing the ML processor can be fabricated on the substrate. The sensing module can then be fabricated on the CMOS circuitry.

[0031] The monolithic integration of the sensing module and the ML processor provides a 3D heterogeneous integration of sensing and processing functions, enabling desirable hardware processing capabilities such as near-field sensing processing, in-memory computing, analog computing, and parallel computing. The monolithically integrated sensing module and ML processor can be used to implement 3D neural network hardware, resulting in increased device density, complex connectivity, and reduced communication loss. In some embodiments, the ML processor can also provide a two-dimensional interface (a cross-section of a 3D neural network) for communicating with a 2D sensor array (e.g., an image sensor array), allowing sensing data generated by the 2D sensor array to be directly input to and processed by the 3D neural network without requiring signal storage or reconfiguration into one-dimensional data (e.g., vectors used as input to conventional 2D neural networks).

[0032] Furthermore, most sensing signals can be viewed as some form of energy (e.g., temperature, mechanical force, photons, vibration, chemicals, electromagnetic waves, etc.) and can be easily converted into an electrical signal (e.g., 0.5 V) using an emerging device. These electrical signals can serve not only as analog data to be processed, but also as a potential source of power to self-power the sensing module and other components of the processing device. In some embodiments, the processing device can thus wake up only in the presence of a sensing signal, enabling event-driven applications to be implemented, further reducing the amount of data collected for ML processing and energy consumption.

[0033] 1A and 1B are schematic diagrams illustrating example processing devices 100a and 100b with integrated sensing and processing capabilities, according to some embodiments of the present disclosure.

[0034] 1A, processing device 100a may include a sensing module 110, a machine learning (ML) processor 120, and a communication module 130. ML processor 120 may further include a pre-processing unit 121, an analog-to-digital conversion (ADC) unit 123, and a machine learning (ML) processing unit 125. Sensing module 110, ML processor 120, and communication module 130 may be integrated onto a single monolithic wafer, such as the monolithic wafer described in connection with FIGS. 7A-9 below.

[0035] The sensing module 110 can include one or more sensor arrays. Each sensor array can include one or more sensors capable of detecting and / or measuring a physical property and generating an electrical signal representative of the physical property. Examples of sensors include image sensors, sound sensors, chemical sensors, pressure sensors, heat sensors, temperature sensors, vibration sensors, microbial fuel cells, electromagnetic sensors, etc. In some embodiments, the multiple sensor arrays of the sensing module 110 can include various types of sensors. In some embodiments, the sensing module 110 can include one or more image sensors, as described in connection with Figures 5A-5B below. In some embodiments, the sensing module 110 can generate analog sensing data in the form of an analog sensing signal (e.g., a voltage signal, a current signal, etc.), such as an analog image signal generated by an image sensor array (e.g., a CMOS image sensor) capable of detecting light and generating an analog image signal representative of the detected light.

[0036] In some embodiments, the sensors of sensing module 110 can obtain enough energy to operate ML processor 120 and / or processing device 100a without requiring an external power source. For example, the electrical signals generated by sensing module 110 can be used to power ML processor 120 and / or processing device 100a.

[0037] The ML processor 120 can process the analog sensing data generated by the sensing module 110 using one or more machine learning models. For example, the preprocessing unit 121 can process the analog sensing data and generate analog preprocessed sensing data. The preprocessing unit 121 can perform any suitable operation on the analog sensing signal to prepare the analog sensing data for subsequent processing by the ML processing unit 125. For example, the preprocessing unit 121 can perform feature extraction on the analog sensing data, and the extracted features of the analog sensing data can be used in subsequent ML processing. As another example, the preprocessing unit 121 can perform dimensionality reduction on the analog sensing data to reduce the amount of data processed in subsequent ML processes. As a further example, the preprocessing unit 121 can perform one or more convolution operations (e.g., a two-dimensional convolution operation, a depthwise convolution operation, etc.) on the analog sensing data. As yet another example, the preprocessing unit 121 can normalize the analog sensing data, rescale and / or resize the analog sensing data, denoise the analog sensing data, etc. In some embodiments, where the analog sensing data includes analog image signals, the pre-processing unit 121 may process the analog sensing data using suitable image processing techniques.

[0038] The pre-processing unit 121 may include one or more crossbar arrays capable of processing analog sensing signals in the analog domain. Each crossbar array may include a plurality of interconnected electrically conductive wires (e.g., row wires, column wires, etc.) and crosspoint devices fabricated at the intersections of the electrically conductive wires. The crosspoint devices may include, for example, memristors, phase-change memory devices, floating gates, spintronic devices, and / or other suitable devices with programmable resistance. In some embodiments, the crossbar array may include one or more crossbar arrays described in connection with Figures 2-3 below.

[0039] As an example, the crossbar array can receive an input voltage signal V and generate an output current signal I. The relationship between the input voltage signal and the output current signal can be expressed as I=VG, where G represents the conductance value of the crosspoint device. In this manner, the input signal is weighted by its conductance at each crosspoint device according to Ohm's law. The weighted currents are output via each bit line and can accumulate according to Kirchhoff's current law. The conductance values ​​of the crosspoint devices can be programmed to values ​​and / or weights representing one or more matrices used to perform the preprocessing of the analog sensing data described above (e.g., feature extraction, dimensionality reduction, convolution, image processing, etc.). The crossbar array can receive as input the analog sensing signals generated by the sensors of the sensing module 110 and generate an analog output signal (e.g., a current signal) representing the preprocessed analog sensing data.

[0040] In some embodiments, the sensing module 110 may include sensors arranged as a two-dimensional (2D) sensor array. Because each sensor may generate an analog sensing signal, the output of the 2D sensor array may be considered a 2D output including the analog sensing signals generated by the sensors (e.g., m×n analog sensing signals generated by m×n sensors). The pre-processing unit 121 may include a three-dimensional (3D) crossbar array including multiple 2D crossbar arrays arranged in a 3D manner. For example, the 2D crossbar arrays may be arranged in different planes (e.g., parallel planes perpendicular to a substrate on which the 3D crossbar array is fabricated). The cross-section of the 3D crossbar array is 2D, allowing the 2D output (e.g., m×n analog sensing signals) generated by the sensing module 110 to be received and / or processed without converting it into one-dimensional data (e.g., vectors representing the sensing signals generated by the sensors). The 3D crossbar array circuit may be and / or include the 3D crossbar array circuit described in connection with FIG. 3 below. In some embodiments, the 3D crossbar array circuit can be fabricated using techniques described in connection with U.S. patent application Ser. No. 16 / 521,975, entitled "Crossbar Array Circuit with 3D Vertical RRAM," the entirety of which is incorporated herein by reference.

[0041] The analog-to-digital converter (ADC) 123 may include any suitable circuitry for converting analog pre-processed sensing data into digital pre-processed sensing data. In some embodiments, the ADC unit 123 may include one or more ADCs 250, which are described in connection with FIG. 2 below.

[0042] The ML processing unit 125 may include circuitry for processing the digital preprocessed sensing data using one or more machine learning models. In some embodiments, the ML processing unit 125 may include a digital signal processor. The ML processing unit 125 may generate predicted outputs by running machine learning models trained using the digital preprocessed sensing data. The predicted outputs may represent, for example, classification results (e.g., class labels assigned to the sensing data), decisions made based on the machine learning models, etc. A machine learning model may refer to a model artifact generated by a processing device using training data that includes known training inputs and corresponding known outputs (correct answers for each training input). The processing device may find patterns in the training data that map known inputs to known outputs (predicted outputs) and provide a machine learning model that captures these patterns.

[0043] The machine learning model may include a machine learning model configured with a single level of linear or nonlinear operations (e.g., a support vector machine), a neural network configured with multiple levels of nonlinear operations, etc. The neural network may include an input layer, one or more hidden layers, and an output layer. The neural network may be trained by adjusting the weights of the neural network according to, for example, a backpropagation learning algorithm. In some embodiments, the crossbar array of the preprocessing unit 121 may implement one or more layers of the neural network. For example, the analog preprocessed sensing data generated by the preprocessing unit 121 may represent the output of the input layer or hidden layer of the neural network.

[0044] Communications module 130 may include any suitable hardware and / or software for facilitating communication between processing device 100a and one or more other computing devices. For example, communications module 130 may include one or more transceivers capable of transmitting and / or receiving RF (radio frequency) signals. Communications module 130 may include components for implementing one or more other wireless transmission protocols (e.g., Wi-Fi, BLUETOOTH, ZIGBEE, cellular, etc.). In some embodiments, communications module 130 may include one or more antennas, which may be integrated into processor wafer 420 or package substrate 410 of FIGS. 4A-4B. Communications module 130 may forward the output of ML processor 120 to another computing device (e.g., a cloud computing device) for further processing. In some embodiments, communications module 130 may further receive instructions from a computing device to perform an operation based on the predicted output (e.g., turn on a display based on the facial recognition result, transmit data to one or more other processing devices, present media content, etc.).

[0045] 1B, processing device 100b may include a sensing module 110, a machine learning (ML) processor 140, and a communication module 130. Sensing module 110 and communication module 130 may be the same as their respective counterparts described in connection with FIG. 1A above.

[0046] The ML processor 140 can process the analog sensing data generated by the sensing module 110 using one or more machine learning models. The ML processor 140 can include a machine learning (ML) processing unit 141 and an ADC 143. The ML processing unit 141 can process the analog sensing data generated by the sensing module 110 using one or more machine learning models to generate an analog predicted output. The analog predicted output can include one or more analog signals.

[0047] In some embodiments, the ML processing unit 141 may include one or more crossbar arrays, each including the crossbar array described in connection with FIG. 2 below. In some embodiments, the ML processing unit 141 may include a 3D crossbar array described in connection with FIG. 3 below. In some embodiments, the crossbar array may implement a neural network that runs a machine learning algorithm. The output signal of the crossbar array may represent the output of the neural network. The neural network may include multiple convolution layers, each performing a specific convolution operation (e.g., 2D convolution, depthwise convolution, etc.). Each layer of the neural network may be implemented using one or more crossbar arrays. For example, one or more first crossbar arrays may implement a first layer (e.g., input layer) of the neural network. The first crossbar array may receive analog sensing data generated by the sensing module 110 as input and perform one or more convolution operations on the analog sensing signals. Performing a convolution operation on the analog sensing data may include convolving different portions of the sensing data with one or more kernels. For example, 2D convolution can be performed by applying a single convolution kernel to analog sensing data. More specifically, the convolution kernel can be used to scan portions of the sensing data that are the same size as the convolution kernel to generate a convolution result. As another example, performing depthwise convolution on sensing data includes convolving each channel of the sensing data with a respective kernel and stacking the convolved outputs. For example, the conductance values ​​of multiple cross-point devices in a first crossbar array can be programmed to values ​​that represent a 2D convolution kernel. An analog sensing signal can be provided as an input signal to the first crossbar array. The first crossbar array can output a current signal that represents the convolution of the analog sensing signal and the 2D convolution kernel.In some embodiments, the crossbar array can store multiple 2D convolution kernels by mapping each of the 2D convolution kernels to multiple crosspoint devices of the first crossbar array. The first crossbar array can output multiple output signals (e.g., current signals) representing the convolution results. The outputs of the first crossbar array can be provided to one or more second crossbar arrays implementing a second layer of a neural network for processing. The outputs of the second crossbar array (e.g., analog current signals) can represent the outputs of the second layer of the neural network. The outputs of the second crossbar array can be provided to one or more third crossbar arrays implementing a subsequent layer (e.g., a second hidden layer) of the neural network for processing. The one or more third crossbar arrays can implement an output layer of the neural network. The outputs of the third crossbar array (e.g., analog current signals) can represent the outputs of the neural network. In some embodiments, the neural network may be implemented using a crossbar array as disclosed in U.S. patent application Ser. No. 16 / 125,454, entitled "Implementation of a Multi-Layer Neural Network Using a Crossbar Array," which is incorporated herein by reference in its entirety.

[0048] The ADC 143 may include any suitable circuitry for converting the analog output of the ML processing unit 141 to a digital output. The digital output may represent a predicted output. In some embodiments, the ADC 143 may include the ADC 250 of FIG. 2.

[0049] The communications module 130 can forward the output of the ML processor 140 to another computing device (e.g., a cloud computing device) for further processing. In some embodiments, the communications module 130 can further receive instructions from the computing device to perform an operation based on the predicted output (e.g., turn on a display based on the facial recognition result, send data to another processing device, present media content, etc.).

[0050] In some embodiments, the processing devices 100a, 100b are self-powered and can operate without an external power source. For example, the sensing module 110 can provide power to components of the processing devices 100a, 100b. The ML processors 120, 140 and their components can be powered and operate as described herein using the analog output generated by the sensing module 110.

[0051] 2 is a diagram illustrating an example crossbar array 200 according to some embodiments of the present disclosure. As shown, crossbar array 200 may include a plurality of interconnected electrically conductive wires, such as one or more row wires 211a, 211b, ..., 211i, ..., 211n and column wires 213a, 213b, ..., 213j, ..., 213m, for an n-row by m-column crossbar array. Crossbar array 200 may further include cross point devices 220a, 220b, ..., 220z, etc. Each crosspoint device can connect row wires and column wires. For example, crosspoint device 220ij can connect row wire 211i and column wire 213j. The number of column wires 213a-m and the number of row wires 211a-n may or may not be the same. The crossbar array 200 may further include word line (WL) logic 205 connected to the crosspoint devices via the row wires 211a-n. The WL logic 205 may include any suitable components (e.g., one or more digital-to-analog converters (DACs), amplifiers, etc.) for applying input signals to selected crosspoint devices via the row wires 211a-n. Each input signal may be a voltage signal, a current signal, etc. The input signals may correspond to the analog sensing signals generated by the sensing module 110 of FIGS. 1A-1B.

[0052] The row wires 211 may include a first row wire 211 a, a second row wire 211 b, ..., 211 i, ..., and an nth row wire 211 n. Each of the row wires 211 a, ..., 211 n may be and / or include any suitable electrically conductive material. In some embodiments, each row wire 211 a-n may be a metal wire.

[0053] The column wires 213 may include a first column wire 213 a, a second column wire 213 b, ..., and an mth column wire 213 m. Each of the column wires 213 a-m may be and / or include any suitable electrically conductive material. In some embodiments, each column wire 213 a-m may be a metal wire.

[0054] Each cross point device 220a-z may be and / or include any suitable device with tunable resistance, such as a memristor, a phase change memory (PCM) device, a floating gate, a spintronic device, a ferroelectric device, an RRAM device, etc.

[0055] Each of the row wires 211 a-n can be connected to one or more row switches 231 (e.g., row switches 231 a, 231 b, ..., 231 n). Each row switch 231 can include any suitable circuit structure capable of controlling the current through the row wires 211 a-n. For example, the row switches 231 can be and / or include CMOS switch circuits.

[0056] Each of the column wires 213a-213m can be connected to one or more column switches 233 (e.g., switches 233a-233m). Each column switch 233a-233m can include any suitable circuit structure capable of controlling the current through the column wires 213a-213m. For example, the column switches 233a-233m can be and / or include CMOS switch circuits. In some embodiments, one or more of the switches 231a-231n and 233a-233m can further provide fault protection, electrostatic discharge (ESD) protection, noise reduction, and / or any other suitable functionality for one or more portions of the crossbar array 200.

[0057] The output sensor 240 may include any suitable components (e.g., one or more TIAs (transimpedance amplifiers) 240a-n) for converting the currents flowing through the column wires 213a-n into output signals. Each TIA 240a-n may convert the currents flowing through a respective column wire into a respective voltage signal. Each ADC 250a-250m may convert the voltage signal generated by the corresponding TIA into a digital output. In some embodiments, the output sensor 240 may further include one or more multiplexers (not shown).

[0058] Programming circuit 260 can program cross point devices 220 selected by switches 231 and / or 233 to appropriate conductance values. For example, programming a cross point device can include applying an appropriate voltage or current signal to the cross point device. The resistance of each cross point device can be electrically switched between a high resistance state and a low resistance state. Setting a cross point device can include switching the resistance of the cross point from a high resistance state to a low resistance state. Resetting a cross point device can include switching the resistance of the cross point from a low resistance state to a high resistance state.

[0059] The crossbar array 200 can perform parallel weighted voltage multiplication and current summation. For example, an input voltage signal can be applied to one or more rows (e.g., one or more selected rows) of the crossbar array 200. The input signal can flow through the crosspoint devices in the row of the crossbar array 200. The conductance of the crosspoint devices can be adjusted to a specific value (also called a "weight"). According to Ohm's law, the input voltage multiplies the crosspoint conductance, generating a current from the crosspoint device. According to Kirchhoff's law, the sum of the currents flowing through the devices on each column generates a current as an output signal, which can be read from the column (e.g., the output of an ADC). According to Ohm's law and Kirchhoff's current law, the input-output relationship of the crossbar array can be expressed as I = VG, where I represents the output signal matrix as a current, V represents the input signal matrix as a voltage, and G represents the conductance matrix of the crosspoint device. In this way, the input signal is weighted by its conductance at each crosspoint device according to Ohm's law. The weighted currents are output through each column wire and can be accumulated according to Kirchhoff's current law, enabling in-memory computing (IMC) through parallel multiplications and additions performed in the crossbar array.

[0060] The crossbar array 200 can be configured to perform vector-matrix multiplication (VMM). A VMM operation can be expressed as Y=XA, where Y, X, and A each represent a respective matrix. More specifically, for example, an input vector X can be mapped to an input voltage V of the crossbar array 200. The matrix A can be mapped to a conductance value G. The output current I can be read and mapped back to an output result Y. In some embodiments, the crossbar array 200 can be configured to implement a portion of a neural network by performing a VMM.

[0061] In some embodiments, the crossbar array 200 can perform convolution operations. For example, performing a 2D convolution on input data may include applying a single convolution kernel to the input signal. Performing a depthwise convolution on input data may include convolving each channel of the input data with a respective kernel corresponding to that channel and stacking the convolved outputs. A convolution kernel may have a specific size defined by multiple dimensions (e.g., width, height, channels, etc.). A convolution kernel may be applied to portions of the input data of the same size to generate an output. The output may be mapped to an element of the convolution result at a position corresponding to the position of the portion of the input data.

[0062] The programming circuitry 260 can program the crossbar array 200 to store convolution kernels for performing 2D convolution operations. For example, the convolution kernel can be converted into a vector and mapped to multiple crosspoint devices of the crossbar array connected to specific bit lines. In particular, the conductance values ​​of the crosspoint devices can be programmed to values ​​representing the convolution kernel. In response to an input signal, the crossbar array 200 can output, via specific bit lines, current signals representing the convolution of the input signal and the 2D convolution kernel. In some embodiments, the crossbar array 200 can store multiple 2D convolution kernels by mapping each of the 2D convolution kernels to a crosspoint device connected to a respective bit line. The crossbar array 200 can output multiple output signals (e.g., current signals) representing the convolution results via column wires 213.

[0063] FIG. 3 is a schematic diagram illustrating an example of a 3D crossbar array circuit 300 according to some embodiments of the present disclosure.

[0064] As shown, the 3D crossbar array circuit 300 can include a first crossbar array 310, a second crossbar array 320, and a third crossbar array 330 arranged in different planes. In some embodiments, the first crossbar array 310, the second crossbar array 320, and the third crossbar array 330 can be arranged in a first plane, a second plane, and a third plane, respectively. The first plane, the second plane, and the third plane can be parallel to one another. In some embodiments, the first plane, the second plane, and the third plane can be perpendicular or parallel to the substrate on which the first crossbar array 310, the second crossbar array 320, and the third crossbar array 330 are formed. Each of the first crossbar array 310, the second crossbar array 320, and the third crossbar array 330 can include one or more 2D crossbar arrays as described in connection with FIG. 2. Although three crossbar arrays are shown in FIG. 3, the 3D crossbar array circuit 300 may include any suitable number of 2D crossbar arrays integrated into the 3D crossbar circuit.

[0065] The first crossbar array 310 may include crosspoint devices 315 connecting a first plurality of wordlines (e.g., WL1_1, WL2_1, WL3_1) and a first plurality of bitlines (e.g., BL1_1, BL2_1, BL3_1). The second crossbar array 320 may include crosspoint devices 325 connecting a second plurality of wordlines (e.g., WL1_2, WL2_2, WL3_2) and a second plurality of bitlines (e.g., BL1_2, BL2_2, BL3_2). The third crossbar array 330 may include crosspoint devices 335 connecting a third plurality of wordlines (e.g., WL1_3, WL2_3, WL3_3) and a third plurality of bitlines (e.g., BL1_3, BL2_3, BL3_3).

[0066] The 3D crossbar array circuit 300 may further include transistors 340. Each transistor 340 may be connected to a respective gate line (GL1, GL2, GL3, etc.) via its gate region. For example, gate line GL1 may be connected to the gate region of a first transistor in the first crossbar array 310, the gate region of a second transistor in the second crossbar array 320, and the gate region of a third transistor in the third crossbar array 330. The source region of each transistor 340 may be connected to a word line. It should be noted that FIG. 3 schematically illustrates the components of the 3D crossbar array circuit 300 and their connections. The schematic diagram shown in FIG. 3 does not represent the physical layout of the components of the 3D crossbar array circuit 300. The components of the 3D crossbar array circuit 300 may be physically arranged in any suitable manner to implement the 3D crossbar arrays described herein. For example, in the physical layout of the 3D crossbar array circuit 300, the transistors 340 may be located on the same layer of the substrate and connected to corresponding gate lines, bit lines, and word lines through vertical vias.

[0067] To select the crosspoint device located at the intersection of WL3_3 and BL3_3, a voltage V G By applying a voltage V to GL3 and grounding the other GLs, the transistor channel on GL3 opens. D The voltage V can be applied to the drain region of the transistor connected to WL3_3, while the drain regions of other transistors located on the same horizontal layer are grounded, so that current flows only through WL3_3. ground can be applied to BL3_3 while maintaining Vs on other BLs that cross WL3_3. Vs can be equal to Vd-Vds, where Vs represents the voltage on the source region of the transistor and Vds represents the voltage drop between the drain and source regions of the transistor. Therefore, only one device on a WL can have both Vs and V groundOther devices on the same WL (WL3_3) will not be programmed due to the lack of voltage difference on those devices.

[0068] Because the crossbar arrays are arranged in a 3D manner in the 3D crossbar array circuit 300, a cross section of the 3D crossbar array circuit 300 can be considered to be a 2D crossbar array. The 3D crossbar array circuit 300 can receive and process 2D inputs (e.g., m×n analog input signals) without storing the 2D inputs or converting the 2D inputs into one-dimensional data (e.g., vectors representing the 2D inputs). For example, the crosspoint devices located at WL1_1, WL1_2, WL1_3, etc., selected as described above, can receive and process analog sensing signals generated by the 2D sensor array.

[0069] 4A and 4B are schematic diagrams illustrating examples of semiconductor devices 400a and 400b that can function as machine learning processors in accordance with some embodiments of the present disclosure.

[0070] 4A , the semiconductor device 400a may include a monolithic wafer 420 and a package substrate 410. The package substrate 410 may include an antenna, a connector, a power supply, etc. In some embodiments, the package substrate 410 does not include CMOS components. The monolithic wafer 420 may be connected to the package substrate 410 via an interconnect layer 430. For example, the interconnect layer 430 may include ball grid array (BGA) bumps. In some embodiments, the package substrate 410 may be connected to a PCB (printed circuit board) substrate (not shown).

[0071] The monolithic wafer 420 can include a sensing module (e.g., the sensing module 110 described in connection with FIGS. 1A-1B) that includes one or more sensor arrays. In some embodiments, the monolithic wafer 420 can include one or more image sensor wafers, as described in connection with FIGS. 5A-5B below.

[0072] Monolithic wafer 420 may further include CMOS elements for implementing ADCs, crossbar arrays, driver ICs (integrated circuits), transceivers, and / or any other suitable components for implementing machine learning processing. In some embodiments, monolithic wafer 420 may include one or more CMOS circuits implementing ML processor 120 of FIG. 1A and / or ML processor 140 of FIG. 1B (e.g., CMOS circuits 600a-b described in connection with FIGS. 6A-6B). Monolithic wafer 420 may include the monolithic wafers described in connection with FIGS. 7A-9 below.

[0073] Referring to FIG. 4B, in some embodiments, multiple monolithic wafers can be stacked on a package substrate 410. As shown, monolithic wafers 420a, 420b, and 420c can be connected to the package substrate 410 via interconnect layers 430a, 430b, and 430c, respectively. In some embodiments, the monolithic wafers 420a, 420b, and 420c can include various types of sensors, sense different signals, and process different sensing signals generated by the sensors. Each of the monolithic wafers 420a-c and interconnect layers 430a-c can be and / or include the corresponding portions described in connection with FIG. 4A (i.e., the monolithic wafer 420 and the interconnect layer 430). While a specific number of wafers are shown in FIGS. 4A and 4B, this is for illustrative purposes only. As described herein, any suitable number of monolithic wafers can be stacked on the package substrate 410.

[0074] 5A and 5B illustrate cross-sectional views of example image sensor wafers 500a and 500b, each of which may also be referred to as a CMOS image sensor (CIS) wafer, according to some embodiments of the present disclosure.

[0075] As shown in FIG. 5A, image sensor wafer 500a can include an image sensor including microlenses 511, color filters 513, and photodiodes 515a. Photodiodes 515a can be fabricated on substrate 505 (e.g., a silicon substrate). Substrate 505 can be and / or include substrate 710 of FIGS. 7A-7F and / or substrate 810 of FIGS. 8A-9. Metal wiring 520a can be located between color filters 513 and photodiodes 515a.

[0076] Incident light can be collected through a microlens 511 and separated into multiple color components by one or more sets of color filters 513. For example, a red filter 513a, a green filter 513b, and a blue filter 513c can separate the red, green, and blue components of the incident light, respectively. The photodiode 515a can accumulate photon charges when exposed to light and convert the charges into electrical signals (voltage signals).

[0077] Referring to FIG. 5B, image sensor wafer 500b includes a backside illumination structure in which photodiode 515b is disposed behind color filter 513 and metal wiring 520b is disposed behind photodiode 515b. Image sensor wafer 500b can be fabricated by fabricating photodiode 515b and metal wiring 520b on a front-side silicon substrate 505, then flipping substrate 505, thinning the back-side substrate, and fabricating color filter 513 and microlens 511 on the backside. As shown, metal wiring 520b is disposed behind photodiode 515b, while metal wiring 520a is disposed in front of photodiode 515a. Light can enter interface 531a in image sensor wafer 500a and interface 531b in image sensor wafer 500b, respectively. In this manner, light can reach photodiode 515b without passing through metal wiring 520b, allowing photodiode 515b to capture more optical signals than photodiode 515a.

[0078] 6A and 6B are schematic diagrams illustrating cross-sectional views of exemplary CMOS circuits 600a and 600b including CMOS-compatible RRAM according to some embodiments of the present disclosure.

[0079] As shown, transistor 603 is fabricated on substrate 601. Transistor 603 may include source region 603a, gate 603b, and drain region 603c. While one transistor is shown in FIG. 6A, this is for illustrative purposes only. In some embodiments, multiple transistors (not shown) may be fabricated on substrate 601. The multiple transistors may be separated by suitable insulator and / or dielectric materials. Substrate 601 may include any suitable material that serves as a substrate for fabricating CMOS circuits, such as silicon (Si), silicon dioxide (SiO), silicon nitride (SiN), aluminum oxide (AlO), aluminum nitride (AlN), etc.

[0080] CMOS circuit 600a may include transistor 603 and interconnect layers 610 fabricated on substrate 601. Each of interconnect layers 610 may provide electrical connection between transistor 603 and / or one or more other devices (e.g., one or more other transistors, one or more other RRAM devices, etc.). Interconnect layers 610 may include, for example, via layers 611, 612, 613, 614, ..., and 615 and metal layers 621, 622, 623, 624, ..., and 625. Although via layers through 615 and pad layers through 625 are shown in Figures 6A-6B for simplicity of illustration, additional via layers and pad layers may be fabricated depending on integration and / or interconnection needs. Each via layer may include one or more metal vias. Each metal via may include a suitable metal material, such as Al, Cu, W, etc. Each metal layer may include one or more metal pads. Each metal pad may include a suitable metal material, such as Al, Cu, W, etc. For example, via layer 611 can include metal vias 611a, 611b, and 611c that can connect to source region 603a, gate 603b, and drain region 603c of transistor 603, respectively. In some embodiments, via layer 611 can include tungsten (W) vias and doped polycrystalline Si (poly-Si) terminals, where the poly-Si terminals can directly contact gate 603b, source region 603a, and drain region 603c of transistor 603. The tungsten vias can directly contact the poly-Si terminals. Other via layers and metal layers above via layer 611 can be fabricated from Cu, W, Al, etc. Metal layer 621 can include metal pads 621a, 621b, and 621c. Metal pads 621a, 621b, and 621c can be connected to metal vias 611a, 611b, and 611c, respectively.

[0081] Each interconnect layer can be fabricated by fabricating a dielectric layer, patterning the dielectric layer, and depositing an appropriate metal on the patterned dielectric layer. The dielectric layer can include any suitable dielectric material, such as silicon nitride (SiN), silicon dioxide (SiO). For example, to fabricate the first via layer 611, a dielectric layer 651 can be fabricated on the substrate 601 and transistor 603. The dielectric layer 651 can be fabricated using any suitable deposition technique. For example, the dielectric layer 651 can be patterned and filled with a metal deposition to fabricate metal vias 611a, 611b, and 611c within the dielectric layer 651. In some embodiments, one or more interconnect layers 610 can be fabricated using a dual damascene manufacturing process, in which the metal vias of a via layer and the metal pads of a metal layer can be deposited and patterned in the same metallization process.

[0082] As shown, pairs of adjacent metal layers can be connected through via layers fabricated between the adjacent metal layers. For example, a first metal layer 621 can be connected to a second metal layer 622 through via layer 612. In particular, metal pad 622a of metal layer 622 can be connected to metal pad 621a of metal layer 621 through metal via 612a. Metal pad 622b of metal layer 622 can be connected to metal pad 621b of metal layer 621 through metal via 612b. As shown in FIG. 6B, metal pad 622c of metal layer 622 can be connected to metal pad 621c of metal layer 621 through metal via 612c.

[0083] Interconnect layer 610 can have a variety of dimensions. The metal pads of metal layers 621, 622, 623, 624, ..., and 625 may have successively increasing sizes. Similarly, the metal vias of via layers 611, 612, 613, 614, ..., and 615 may have successively increasing sizes. For example, CMOS circuit 600a-b may be part of a 65 nm technology node. The width and spacing of the metal pads of metal layer 621 may be approximately 90 nm. The width and spacing of the metal pads of metal layers 622 and 623 may be approximately 100 nm. The width and spacing of the metal pads of metal layer 625 may be approximately 400 nm.

[0084] The RRAM device 640 can be fabricated during the fabrication of the interconnect layer 610. In this manner, the RRAM device 640 is referred to as a CMOS-compatible RRAM device. For example, one or more first interconnect layers 610a may be fabricated on the transistor 603 and / or the substrate 601. The RRAM device 640 may be fabricated on a metal pad or a metal via of the top interconnect layer of the first interconnect layer 610a. One or more second interconnect layers 610b may then be fabricated on the RRAM device 640 and the first interconnect layer 610a. More specifically, for example, a metal pad or a metal via of the bottom interconnect layer of the second interconnect layer 610b may be fabricated on the RRAM device 640 and directly contact the RRAM device 640. In some embodiments, as shown in FIG. 6A , the first interconnect layer 610a may include a via layer 611 and a metal layer 621. The metal layer 621 may be considered the top interconnect layer of the first interconnect layer 610a. The RRAM device 640 can be fabricated on a metal pad 621c of the metal layer 621. The RRAM device 640 is connected to a drain region 603c of the transistor 603 through a metal pad 621c of the metal layer 621 and a metal via 611c of the via layer 611. A metal via 612c of the via layer 612 can be fabricated on the RRAM device 640 and connected to a bit line (e.g., column wires 213a-m in FIG. 2) of a circuit including the RRAM device 640. The metal vias 612a and 612b of the via layer 612 can be fabricated on the metal pads 621a and 621b, respectively. The metal layer 612 can be considered the lowest interconnect layer of the second interconnect layer 610b. Second interconnect layer 610b can include one or more metal layers and / or via layers (eg, metal layers 622, 623, 624, and 625 and via layers 613, 614, and 615) fabricated on metal layer 612.

[0085] In some embodiments, as shown in FIG. 6B , RRAM device 640 can be fabricated on metal pad 622c of metal layer 622. Via layer 613 can be fabricated on RRAM device 640. In particular, metal via 613c of via layer 613 is fabricated on RRAM device 640 and directly contacts RRAM device 640. In such embodiments, first interconnect layer 610a can include via layer 611, metal layer 621, via layer 612, and metal layer 622. Metal layer 622 can be considered the top interconnect layer of first interconnect layer 610a. Second interconnect layer 610b can include via layers 613, 614, and 615 and metal layers 623, 624, and 625. Via layer 613 can be considered the bottom interconnect layer of second interconnect layer 610b.

[0086] 6A and 6B may be the same, fabricating the first interconnect layer 610a in FIG. 6B involves more steps than fabricating the first interconnect layer 610a in FIG. 6A, while fabricating the second interconnect layer 610b in FIG. 6B involves fewer steps than fabricating the second interconnect layer 610b in FIG. 6A. RRAM device 640 is connected to drain region 603c of transistor 603 through metal pad 622c in metal layer 622, metal via 612c in via layer 612, metal pad 621c in metal layer 621, and metal via 611c in via layer 611. Metal via 613c in via layer 613 is fabricated over RRAM device 640 and can be connected to a circuit bit line (e.g., column line 213 in FIG. 2). Metal vias 613a and 613b of via layer 613 can be fabricated on metal pads 622a and 622b, respectively.

[0087] Although particular interconnect layers (e.g., metal layers and via layers) are shown in Figures 6A-6B, this is for illustrative purposes only. CMOS circuits 600a-b may include any suitable number of interconnect layers for implementing various integrated circuits. First interconnect layer 610a and second interconnect layer 610b may include any suitable number of interconnect layers. For example, in some embodiments, RRAM device 640 may be fabricated on metal layer 623 or other metal layers. In some embodiments, RRAM device 640 may be connected to source region 603a of transistor 603. In such embodiments, RRAM device 640 may be fabricated on a metal pad and / or metal via connected to source region 603a.

[0088] 7A and 7B are schematic diagrams illustrating a cross-sectional view 700a and a top view 700b, respectively, of a monolithic wafer with integrated sensing and ML processing capabilities according to some implementations of the present disclosure. The monolithic wafer can be and / or include the monolithic wafer described in connection with FIGS. 4A-4B above.

[0089] Referring to FIG. 7A, a monolithic wafer can include a sensing module 720 and an ML processor 730 fabricated on a substrate 710. The substrate 710 can be and / or include any suitable material suitable for fabricating the components of the sensing module and ML processor described herein. For example, the substrate 710 can include a silicon substrate or any other suitable substrate for fabricating the CMOS circuits and sensing module described herein. The sensing module 720 and the ML processor 730 can be fabricated on the same side of the substrate 710 and on two non-overlapping portions of the substrate 710. For example, as shown in FIG. 7B, the sensing module 720 can be fabricated on a first portion (e.g., a central portion) of the substrate 710. The ML processor 730 can be fabricated on a second portion of the substrate 710 surrounding the first portion. This arrangement of the sensing module 720 and the ML processor 730 can reduce the routing distance between the sensing module 720 and the ML processor 730. It should be noted that the sensing module 720 and the ML processor 730 may be arranged side-by-side in any other suitable manner on the substrate 710.

[0090] The sensing module 720 may include multiple sensors (e.g., a sensor array), interconnect layers (e.g., metal wiring), and / or any other suitable components for implementing the sensing module 110 of FIGS. 1A-1B. For example, the sensing module 720 may include multiple photodiodes, color filters, and microlenses for implementing an image sensor array. In some embodiments, the sensing module 720 may include the CIS wafers 500a and / or 500b described in connection with FIGS. 5A-5B. The ML processor 730 may include transistors, RRAM devices, interconnect layers, and / or any other suitable components for implementing the ML processors 120 and / or 140 of FIGS. 1A-1B. In some embodiments, the ML processor 730 may include one or more crossbar arrays for implementing various functional components of the ML processors 120 and / or 140 (e.g., the crossbar arrays described in connection with FIGS. 2-3). Each crossbar array may include one or more CMOS circuits 600a-b described in connection with FIGS. 6A-6B.

[0091] 7C-7I are schematic diagrams illustrating cross-sectional views of structures 700c, 700d, 700e, 700f, 700g, 700h, and 700i for fabricating monolithic wafers as described in connection with FIGS. 7A-7B.

[0092] As shown in FIG. 7C, transistors 731 and 733 and photodiodes 721a, 721b, . . . , 721n of ML processor 730 can be fabricated on substrate 710 (eg, surface 711 of substrate 710). Each photodiode 721a-721n may be a two-terminal device including a p-n junction capable of converting photons into electrical current. For example, each photodiode 721a-721b may include a p-type doped region and an n-type doped region (e.g., p-type doped region 7211a and n-type doped region 7213a of photodiode 721a, p-type doped region 7211b and n-type doped region 7213b of photodiode 721b, p-type doped region 7211n and n-type doped region 7213n of photodiode 721n, etc.). In some embodiments, one or more photodiodes 721a-721n may have a p-i-n structure including an i representing an undoped intrinsic region (not shown) between the p-type and n-type doped regions. When a bias is applied to the photodiode, the current output can be controlled to provide thresholding, a linear response, or a nonlinear response. In particular, photodiodes 721a-721n can be reverse biased or slightly forward biased to produce an output current that is a linear function of input light intensity. Transistor 731 can include a source region 731a, a gate region 731b, and a drain region 731c. Transistor 733 can include a source region 733a, a gate region 733b, and a drain region 733c. While a specific number of transistors and photodiodes are shown in FIG. 7C, this is for illustrative purposes only. ML processor 730 can include any suitable number of transistors fabricated on substrate 710. Sensing module 720 can include any suitable number of photodiodes fabricated on substrate 710. The transistors can be fabricated in a portion of substrate 710 that surrounds the photodiodes.

[0093] As shown in FIG. 7D , one or more interconnect layers 750a (also referred to as a “first interconnect layer”) can be fabricated over the transistors of the ML processor 730 and the photodiodes of the sensing module 720. The first interconnect layer 750a can be and / or include the metal layers and / or via layers described in connection with FIGS. 6A-6B above. As shown in FIG. 7D , a first via layer 741 can be fabricated over the transistor 731, the transistor 733, and the photodiodes 721a-n. The first via layer 741 can include multiple metal vias 741a, 741b, 741c, ..., 741l. The metal vias 741a, 741b, and 741c can be fabricated over the source region 731a, the gate region 731b, and the drain region 731c of the transistor 731, respectively. The metal vias 741d and 741e can be fabricated over the photodiode 721a. Metal vias 741f and 741g can be fabricated over photodiode 721b. Metal vias 741h and 741i can be fabricated over photodiode 721n. Metal vias 741j, 741k, and 741l can be fabricated over source region 733a, gate region 733b, and drain region 733c of transistor 733, respectively. A first metal layer 751 can be fabricated over first via layer 741. First metal layer 751 can include metal pads 751a-751l, which can be fabricated over metal vias 741a-741l, respectively. In some embodiments, the metal pads of first metal layer 751 can directly contact the metal vias of first metal via layer 741. Each of interconnect layers 750a can be fabricated by fabricating a dielectric layer (e.g., dielectric layers 761 and 763), patterning the dielectric layer, and depositing an appropriate metal on the patterned dielectric layer. In some embodiments, the via layer 741 and the metal layer 751 may be fabricated using a dual damascene manufacturing process.

[0094] One or more RRAM devices can be fabricated on the top interconnect layer of the first interconnect layer 750a. For example, RRAM devices 735 and 737 can be fabricated on metal pads 751c and 751h, respectively. It should be noted that RRAM devices 735 and 737 can be fabricated on any suitable interconnect layer described herein. RRAM devices 735 and 737 may or may not be fabricated on the same interconnect layer.

[0095] As shown in FIG. 7E, one or more interconnect layers 750b (also referred to as "second interconnect layers") can be fabricated over RRAM devices 735 and 737. Interconnect layers 750a and 750b are also referred to as interconnect layer 750. Each of second interconnect layers 750b can be a metal layer and / or a via layer described in connection with FIGS. 6A-6B. For example, second interconnect layer 750b can include via layers 743, ..., 745 and metal layers 753, ..., 755. Metal pads 755a-755l of metal layer 755 can be connected to metal pads 753a-753l of metal layer 753 through metal vias 745a-745l, respectively, of via layer 745. Metal pads of metal layers 753 and 751 can be connected through metal vias of via layer 743. Although a specific number of interconnect layers is shown in FIG. 7E, this is for illustrative purposes only. Second interconnect layer 750b can include any suitable number of interconnect layers. Each of second interconnect layers 750b can be fabricated by fabricating a dielectric layer (e.g., dielectric layers 765, 767, 769, and 771), patterning the dielectric layer, and depositing an appropriate metal on the patterned dielectric layer. In some embodiments, the via layer and metal layer (e.g., via layer 743 and metal layer 753) can be fabricated using a dual damascene manufacturing process. In some embodiments, one or more through-silicon via (TSV) connections (not shown) can be fabricated with interconnect layer 750 to provide connections between the monolithic wafer and a packaging substrate.

[0096] As shown in FIG. 7F, color filters 723a, 723b, ..., and 723n can be fabricated on second interconnect layer 750b and / or interconnect layer 750. In some embodiments, the set of color filters 723a-n can separate incident light into multiple color components. For example, color filters 723a, 723b, and 723n can separate red, green, and blue components of incident light, respectively. Microlenses 725a, 725b, ..., 725n can be fabricated on color filters 723a-n, respectively. Although two sets of microlenses and color filters are shown in FIG. 7F, this is for illustrative purposes only. Any suitable number of color filters and microlenses can be fabricated on interconnect layer 750.

[0097] In some embodiments, the semiconductor structure 700e of FIG. 7E can be inverted to fabricate the backside illuminated structure described in connection with FIG. 5B above. For example, as shown in FIG. 7G, the semiconductor structure 700e can be inverted to expose the backside (e.g., surface 713) of the substrate 710. As shown in FIG. 7H, the substrate 710 can be thinned to an appropriate thickness to allow incident light to reach the photodiode. The thinned backside substrate is also referred to as substrate 710a. As shown in FIG. 7I, color filters 723a-n and microlenses 725a-n can be fabricated on surface 713a of the substrate 710a. Surfaces 713a and 711 correspond to two opposite sides of the substrate 710.

[0098] 8A-8B are schematic diagrams illustrating cross-sectional views 800a and 800b of an exemplary monolithic wafer with integrated sensing and processing capabilities according to some embodiments of the present disclosure.

[0099] 8A, the ML processor 820 can be fabricated on a substrate 810. The substrate 810 can be a single crystal silicon substrate in some embodiments. The substrate 810 can then be flipped over, and the sensing module 830 can be fabricated on the second side of the substrate 810. In this manner, the ML processor 820 and the sensing module 830 can be fabricated on different (e.g., opposite) sides of the substrate 810 using double-sided and / or dual-sided wafer processing techniques. The substrate 810 can be a double-sided polished semiconductor wafer (e.g., a silicon wafer) having two polished surfaces and / or sides suitable for fabricating the CMOS circuits and sensing modules described herein. For example, multiple CMOS circuits (e.g., CMOS circuits 600a and / or 600b of FIGS. 6A-6B) can be fabricated on a first side of the substrate 810 to implement the crossbar array and other components of the ML processors 120 and / or 140 of FIGS. 1A-1B. The sensing module 830 can be fabricated on a second side of the substrate 810 shown in FIG. 8B. The sensing module 830 can include one or more sensors fabricated on the second side of the substrate 810. In some embodiments, the sensing module 830 can include photodiodes, color filters, one or more interconnect layers, microlenses, and any other suitable components for implementing an image sensor array as described herein. For example, the sensing module 830 can include one or more sensor wafers 500a-b described in connection with FIGS. 5A-5B.

[0100] 8C, 8D, 8E, and 8F are diagrams illustrating cross-sectional views of exemplary structures 800c, 800d, 800e, and 800f for fabricating a monolithic wafer as described in connection with FIGS. 8A-8B according to some embodiments of the present disclosure.

[0101] To fabricate a monolithic wafer, the transistors of the ML processor 820 and the photodiodes of the sensing module 830 may be first fabricated on the substrate 810 because fabrication of the transistors and photodiodes involves high-temperature processes (e.g., thermal oxidation, ion implantation, diffusion, etc.). For example, as shown in FIG. 8C, a transistor 821 may be fabricated on the substrate 810. The transistor 821 may include a source region 821a, a gate 821b, and a drain region 821c. The substrate 810 may be inverted to expose a second surface (e.g., backside) of the substrate 810. As shown in FIG. 8D, one or more photodiodes 831a-831n may be fabricated on the second surface of the substrate 810. While a specific number of transistors and photodiodes are shown in FIGS. 8C-8D, this is for illustrative purposes only. Any suitable number of transistors and photodiodes may be fabricated on the substrate 810 to implement the ML processors described herein.

[0102] Referring to FIG. 8E, one or more interconnect layers (also referred to as “first plurality of interconnect layers”) can be fabricated on transistor 821. The first plurality of interconnect layers can include metal layers 851, 853, ..., 855 and via layers 841, 843, ..., 845. Each of the first plurality of interconnect layers can be the interconnect layer described in connection with FIGS. 6A-6B above. Via layer 841 can include metal vias 841a, 841b, and 841c connected to source region 821a, gate 821b, and drain region 821c, respectively, of transistor 821. Metal layer 851 can include metal pads 851a, 851b, and 851c connected to metal vias 841a, 841b, and 841c, respectively. The first plurality of interconnect layers can include any suitable number of metal layers and via layers. RRAM device 823 can be fabricated during the fabrication of the interconnect layers. For example, RRAM device 823 can be fabricated on metal pad 851c of metal layer 851. Metal via 843c of via layer 843 can be fabricated on RRAM device 823. Metal vias 843a and 843b of via layer 843 can be connected to metal pads 851a and 851b, respectively, of metal layer 851. In some embodiments, RRAM device 823 can be fabricated on metal pads and / or metal vias connected to the source region of transistor 821.

[0103] 8F, a plurality of interconnect layers (also referred to as a "second plurality of interconnect layers") can be fabricated over the photodiodes 831a-n. The second plurality of interconnect layers can include, for example, via layers 861, 863, ..., 865 and metal layers 871, 873, ..., 875. The via layer 861 can include metal vias 861a, 861b, ..., 861n fabricated over the photodiodes 831a-n. The metal layer 871 can include metal pads 871a, 871b, ..., 871n fabricated over the metal vias 861a, 861b, ..., 861n, respectively. The metal pads 875a, 875b, ..., 875n can be connected to metal pads 873a, 873b, ..., 873n of the metal layer 873 through metal vias 865a, 865b, ..., 865n of the via layer 865. Metal pads 873a, 873b, ..., 873n can be connected to metal pads 871a, 871b, ..., 871n of metal layer 871 through metal vias 863a, 863b, ..., 863n of via layer 863. The second plurality of interconnect layers can include any appropriate number of metal layers and via layers. Color filters 833a-n can be fabricated on the second plurality of interconnect layers (e.g., metal pads 875a, 875b, ..., and 875n of metal layer 875). Microlenses 835a-n can be fabricated on the color filters 833a-n, respectively. In some embodiments, through-silicon vias (not shown) can be fabricated to connect ML processor 820 to sensing module 830. In some embodiments, the sensing module can be fabricated on ML processor 820.

[0104] Figure 9A is a schematic diagram illustrating a cross-sectional view of an exemplary monolithic wafer 900a with integrated sensing and processing capabilities according to some embodiments of the present disclosure, and Figure 9B is a schematic diagram illustrating an example of the monolithic wafer shown in Figure 9A.

[0105] As shown in FIGS. 9A-9B, the ML processor 820 can be fabricated on the substrate 810 as described in connection with FIG. 8A above. A sensing module 930 can be fabricated on the ML processor 820. The sensing module 930 can include one or more photodiodes fabricated on the interconnect layers of the ML processor 820, such as, for example, one or more amorphous silicon (a-Si) photodiodes, polysilicon (poly-Si) photodiodes, and / or any other suitable photodiodes and / or sensors that can be fabricated on the CMOS circuitry described herein. The sensing module 930 can further include color filters, microlenses, and / or any other suitable components fabricated on the photodiodes. Stacking the sensing module on the ML processor can further reduce the size of the monolithic wafer by stacking a-Si photodiodes, poly-Si photodiodes, etc. on the ML processor.

[0106] For ease of explanation, the methods of the present disclosure are depicted and described as a series of steps. However, steps in accordance with the present disclosure can be performed in various orders and / or simultaneously with other steps not shown and described herein. Moreover, not all disclosed steps may be required to implement a method in accordance with the disclosed subject matter. In addition, those skilled in the art will understand and appreciate that the method can alternatively be represented as a series of interrelated states via a state diagram or events.

[0107] As used herein, the terms "approximately," "about," and "substantially" can mean within normal tolerances in the art, e.g., within two standard deviations of the mean, in some embodiments, within ±20% of the target dimension, in some embodiments, within ±10% of the target dimension, in some embodiments, within ±5% of the target dimension, in some embodiments, within ±2% of the target dimension, in some embodiments, within ±1% of the target dimension, and in some embodiments, within ±0.1% of the target dimension. The terms "approximately" and "about" can include the target dimension. Unless otherwise stated or apparent from the context, all numerical values ​​described herein are modified by the term "about."

[0108] As used herein, ranges include all values ​​within that range. For example, the range of 1 to 10 can include any number of the digits 1, 2, 3, 4, 5, 6, 7, 8, 9, and 10, combinations of digits, subranges, and fractions thereof.

[0109] In the foregoing description, many details are set forth. However, it will be apparent that the disclosure may be practiced without these specific details. In some instances, well-known structures and devices are shown in block diagram form, rather than in detail, in order to avoid obscuring the disclosure.

[0110] As used herein, terms such as "first," "second," "third," "fourth," etc. are meant as labels to distinguish between different elements and do not necessarily have an ordinal meaning according to their numerical designation.

[0111] The word "example" or "exemplary" is used herein to mean serving as an example, instance, or illustration. Any aspect or design described herein as "example" or "exemplary" is not necessarily to be construed as preferred or advantageous over other aspects or designs. Rather, use of the word "example" or "exemplary" is intended to present concepts in a concrete manner. The term "or" as used in this application is intended to mean an inclusive "or" rather than an exclusive "or." That is, unless otherwise specified or clear from the context, "X includes A or B" is intended to mean any of the natural inclusive permutations. That is, if X includes A; if X includes B; or if X includes both A and B, then X includes A or B, as in any of the foregoing examples. Additionally, the articles "a" and "an," as used in this application and the appended claims, should be construed generally to mean "one or more" unless otherwise specified or clear from the context that the singular form is intended. Throughout this specification, a reference to an "implementation" or "one implementation" means that a particular feature, structure, or characteristic described in connection with that implementation is included in at least one implementation. Thus, the appearances of the phrase "implementation" or "one implementation" in various places throughout this specification are not necessarily all referring to the same implementation.

[0112] As used herein, when an element or layer is referred to as being "on" another element or layer, the element or layer can be located directly on the other element or layer, or there can be intervening elements or layers. In contrast, when an element or layer is referred to as being "directly on" another element or layer, there are no intervening elements or layers.

[0113] Although many variations and modifications of the disclosure will no doubt become apparent to those skilled in the art after reading the foregoing description, it should be understood that while particular embodiments have been shown and described by way of illustration, they are not intended to be limiting. Accordingly, references to details of various embodiments are not intended to limit the scope of the claims, which will recite only those features that are deemed to be disclosed.

[0114] [CROSS-REFERENCE TO RELATED APPLICATIONS] This application claims the benefit of U.S. patent application Ser. No. 18 / 052,071, filed November 2, 2022, entitled "Integrated Sensing and Machine Learning Processing Devices," which is a continuation-in-part of U.S. patent application Ser. No. 17 / 932,432, filed September 15, 2022, entitled "Integrated Sensing and Machine Learning Processing Devices," each of which is incorporated by reference herein in its entirety.

Claims

1. It is a semiconductor device, A sensing module configured to generate multiple analog sensing signals, Machine learning (ML) processor and Equipped with, The aforementioned ML processor is One or more crossbar arrays configured to process the analog sensing signal and generate analog preprocessed sensing data, An analog-to-digital converter (ADC) configured to convert the aforementioned analog preprocessing sensing data into digital preprocessing sensing data, Includes a machine learning processing unit configured to process the digital preprocessed sensing data using one or more machine learning models, The sensing module and the ML processor are manufactured on a single wafer. The sensing module is manufactured on the first portion of the wafer, The ML processor is a semiconductor device manufactured on a second portion of the wafer that surrounds the first portion of the wafer.

2. The semiconductor device according to claim 1, wherein the sensing module and the ML processor are manufactured on the same side of the wafer.

3. The multiple transistors of the ML processor and the multiple sensors of the sensing module are manufactured on the substrate of the wafer. The semiconductor device according to claim 2, further comprising the plurality of transistors and the plurality of interconnection layers manufactured on the plurality of sensors.

4. The aforementioned multiple sensors include multiple photodiodes, The first metal via, second metal via, and third metal via of the first interconnection layer are connected to the source region, gate region, and drain region of the first transistor of the plurality of transistors, respectively. The semiconductor device according to claim 3, wherein the fourth and fifth metal vias of the first interconnection layer are connected to the first photodiode of the plurality of photodiodes.

5. The resistive random access memory (RRAM) device of the ML processor is manufactured on a metal pad or metal via of one of the multiple interconnection layers, The semiconductor device according to claim 4, wherein the RRAM device is connected to the first transistor via the first interconnection layer.

6. The semiconductor device according to claim 5, wherein the set of microlenses and color filters of the sensing module is manufactured on the plurality of interconnection layers.

7. The semiconductor device according to claim 2, wherein the sensing module is manufactured on the ML processor.

8. The semiconductor device according to claim 1, wherein the sensing module and the ML processor are manufactured on different sides of the wafer.

9. The sensing module includes an image sensor array, The semiconductor device according to claim 1, wherein the plurality of analog sensing signals include a plurality of analog image signals.

10. The analog preprocessed sensing data corresponds to a plurality of features extracted from the analog sensing signal, The semiconductor device according to claim 1, wherein the machine learning processing unit performs machine learning using the extracted features.

11. Further equipped with a packaging substrate, The semiconductor device according to claim 1, wherein the wafer is connected to the packaging substrate via an interconnection layer.

12. The semiconductor device according to claim 1, wherein the ML processor is powered using the analog sensing signal.

13. Equipped with a transceiver, The aforementioned transceiver, The semiconductor device according to claim 1, configured to transmit a prediction output generated by the machine learning processing unit based on one or more machine learning models to a computing device, and to receive instructions from the computing device for performing an operation based on the prediction output.

14. The semiconductor device according to claim 1, wherein the analog preprocessing sensing data represents the convolution of the analog sensing signal and the kernel.

15. The semiconductor device according to claim 14, wherein the conductance values ​​of a plurality of crosspoint devices of one or more crossbar arrays are programmed to represent the kernel.

16. The sensing module includes a two-dimensional sensor array, The semiconductor device according to claim 1, wherein the plurality of crosspoint devices of the one or more crossbar arrays are configured to receive the analog sensing signals generated by the two-dimensional sensor array as input.

17. The semiconductor device according to claim 16, wherein the one or more crossbar arrays include a plurality of crossbar arrays arranged in a plurality of different planes.

18. It is a semiconductor device, A sensing module configured to generate multiple analog sensing signals, A machine learning processor configured to generate a predictive output by processing the plurality of analog sensing signals using one or more machine learning models, Equipped with, The aforementioned machine learning processor, Multiple crossbar arrays configured to generate multiple analog outputs representing the predicted output, Includes an analog-to-digital conversion unit configured to convert the plurality of analog outputs representing the predicted output into a plurality of digital signals representing the predicted output, The sensing module and the machine learning processor are manufactured on a wafer, and the sensing module is manufactured on a first portion of the wafer. The ML processor is a semiconductor device manufactured on a second portion of the wafer that surrounds the first portion of the wafer.

19. Further equipped with a packaging substrate, The semiconductor device according to claim 18, wherein the wafer is connected to the packaging substrate via an interconnection layer.

Images