Integrated sensing and machine learning processing device

Abstract

The invention provides a semiconductor device with integrated sensing and processing functions. The semiconductor device includes a sensing module configured to generate a plurality of analog sensing signals and a machine learning (ML) processor. The sensing module and the ML processor are manufactured on a single crystal wafer. The ML processor includes a crossover array that processes the analog sense signal to generate analog pre-processed sense data; an analog-to-digital converter (ADC) for converting the analog pre-processed sensing data into digital pre-processed sensing data; and a machine learning processing unit that processes the digitally pre-processed sensing data using one or more machine learning models.

Description

Cross-references

[0001] This application claims priority to U.S. patent application No. 18 / 052,071, filed on November 2, 2022, entitled “INTEGRATED SENSING AND MACHINE LEARNING PROCESSING DEVICE,” which is a continuation-in-part of U.S. patent application No. 17 / 932,432, filed on September 15, 2022, entitled “INTEGRATED SENSING AND MACHINE LEARNING PROCESSING DEVICE,” each of which is incorporated in its entirety into this application. Technical Field

[0002] Embodiments of the present disclosure generally relate to a computing device, and more particularly, to an integrated sensing and machine learning processing device. Background Art

[0003] Machine learning (ML) is currently widely used in fields such as face recognition, speech recognition, natural language processing, and image processing. ML typically involves analyzing large amounts of sensor data based on complex machine learning models. Traditional edge devices (local devices close to sensors that collect sensor data) lack the computing power to perform such analysis. Therefore, the sensor data generated by the sensor may need to be digitized and transmitted to a remote computing device (e.g., a data center) for processing. This may involve the digitization of large amounts of data and may require advanced communication capabilities, a large amount of energy, and time for transmitting digitized sensor data. Transmitting raw sensor data from sensors to remote devices may raise privacy issues, and encrypting raw sensor data for secure data transmission may further increase the computing cost required for ML. In addition, some applications (e.g., medical applications using ML) may require real-time data processing. Therefore, there is a demand for running machine learning models locally on edge devices. However, traditional edge devices cannot provide integrated sensing and processing capabilities for local extraction of information and features from analog sensor data provided by local sensors and for ML processing. Summary of the invention

[0004] The following is a brief summary of the present disclosure, which is used to provide a basic understanding of some aspects of the present disclosure. The summary is not an extensive overview of the present disclosure. The summary is not intended to identify the key or important elements of the present disclosure, nor is it intended to illustrate any scope of a particular implementation of the present disclosure or any scope of the claims. The sole purpose of the summary is to present some concepts of the present disclosure as a language simplification of a more detailed description presented subsequently.

[0005] The present disclosure provides a semiconductor device that can be used as an integrated sensing and machine learning processing function. The semiconductor device may include a sensing module configured to generate multiple analog sensing signals and a machine learning (ML) processor. The ML processor includes one or more crossbar arrays that process the analog sensing signals to generate analog pre-processed sensing data; an analog-to-digital converter (ADC) configured to convert the analog pre-processed sensing data into digital pre-processed sensing data; and a machine learning processing unit configured to process the digital pre-processed sensing data using one or more machine learning models. The sensing module and the ML processor are manufactured on a single chip.

[0006] In some embodiments, the sensing module and the ML processor are fabricated on the same side of the wafer.

[0007] In some embodiments, the sensing module is fabricated on a first portion of the wafer, and the ML processor is fabricated on a second portion of the wafer surrounding the first portion of the wafer.

[0008] In some embodiments, the plurality of transistors of the ML processor and the plurality of sensors of the sensing module are fabricated on a substrate of the wafer. The semiconductor device further comprises a plurality of interconnect layers fabricated on the plurality of transistors and the plurality of sensors.

[0009] In some embodiments, the first metal via, the second metal via, and the third metal via in the first interconnect layer are respectively connected to the source region, the gate region, and the drain region of the first transistor in the plurality of transistors. In some embodiments, the fourth metal via and the fifth metal via in the first interconnect layer are connected to the first photodiode in the plurality of photodiodes.

[0010] In some embodiments, the RRAM device of the ML processor may be fabricated on a metal pad or metal via on an interconnect layer among the plurality of interconnect layers. In some embodiments, the RRAM device is connected to a first transistor through the first interconnect layer.

[0011] In some embodiments, the sensing module includes a microlens and a set of color filters. The microlens and the color filters of the sensing module are manufactured on the plurality of interconnection layers.

[0012] In some embodiments, the sensing module is fabricated on the ML processor.

[0013] In some embodiments, the sensing module and the ML processor are fabricated on different sides of the wafer.

[0014] In some embodiments, the sensor module includes an image sensor array. The plurality of analog sensor signals include a plurality of analog image signals.

[0015] In some embodiments, the simulated pre-processed sensor data corresponds to a plurality of features extracted from the simulated sensor signal. The machine learning processing unit performs machine learning using the extracted features.

[0016] In some embodiments, the semiconductor device further includes a packaging substrate, wherein the wafer is connected to the packaging substrate through an interconnect layer.

[0017] In some embodiments, the ML processor is powered using the analog sensor signal.

[0018] In some embodiments, the semiconductor device further includes a transceiver configured to: send a prediction output generated by the machine learning processing unit based on the one or more machine learning models to a computing device; and receive instructions from the computing device to perform an execution operation based on the prediction output.

[0019] In some embodiments, the analog pre-processed sensor data represents a convolution of an analog sensor signal and a convolution kernel.

[0020] In some embodiments, conductance values ​​of a plurality of cross-point devices in the one or more crossbar arrays are programmed to represent a value of the convolution kernel.

[0021] In some embodiments, the sensing module includes a two-dimensional sensor array, wherein a plurality of cross-point devices in the one or more cross-point arrays are configured to receive as input an analog sensing signal generated by the two-dimensional sensor array.

[0022] In some embodiments, the one or more crossbar arrays include a plurality of crossbar arrays located in a plurality of different planes.

[0023] According to one or more aspects of the present disclosure, a semiconductor device includes: a sensing module configured to generate a plurality of analog sensing signals; and a machine learning (ML) processor configured to generate a prediction output by processing the analog sensing signals using one or more machine learning models. The machine learning processor includes: a plurality of crossbar arrays configured to generate a plurality of analog outputs representing the prediction outputs; and an analog-to-digital conversion unit configured to convert the plurality of analog outputs representing the prediction outputs into a plurality of digital signals representing the prediction outputs, wherein the sensing module and the ML processor are manufactured on a wafer.

[0024] In some embodiments, the semiconductor device further includes a packaging substrate, wherein the wafer is connected to the packaging substrate through an interconnect layer. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The present disclosure will be more fully understood from the detailed description given below and the accompanying drawings of various embodiments of the present disclosure. However, the accompanying drawings should not be used to limit the present disclosure to specific embodiments, but are only for explanation and understanding.

[0026] Figure 1A and 1B is a schematic diagram showing an example of a processing device with integrated sensing and processing capabilities according to some embodiments of the present disclosure.

[0027] Figure 2 is a schematic diagram showing an example of a crossbar array according to some embodiments of the present disclosure.

[0028] Figure 3 is a schematic diagram showing an example of a three-dimensional cross array according to some embodiments of the present disclosure.

[0029] Figure 4A and 4B is a schematic diagram showing an example of a semiconductor device that can be used as a machine learning processor in some embodiments of the present disclosure.

[0030] Figure 5A and 5B is a schematic diagram showing a cross-sectional view of an image sensor wafer according to some embodiments of the present disclosure.

[0031] Fig. 6A and 6B is a schematic diagram showing a cross-sectional view of an example of a CMOS circuit including CMOS-compatible RRAMs according to some embodiments of the present disclosure.

[0032] Fig. 7A and 7B are schematic diagrams showing a cross-sectional view and a top view, respectively, of a single wafer with integrated sensing and ML processing capabilities according to some embodiments of the present disclosure.

[0033] Figure 7C-7I It shows the combination Figure 7A-7B A schematic diagram of a cross-sectional view of the structure used to manufacture a single wafer.

[0034] Fig. 8A and 8B is a schematic diagram showing a cross-sectional view of an example of a single wafer with integrated sensing and processing capabilities according to some embodiments of the present disclosure.

[0035] Figure 8C , 8D 8E and 8F are diagrams showing the combination of some embodiments of the present disclosure. Figures 8A-8B A schematic diagram of a cross-sectional view of an example of the structure of the manufactured single wafer.

[0036] Fig. 9A is a schematic diagram showing a cross-section of an example of a single wafer with integrated sensing and processing capabilities according to some embodiments of the present disclosure.

[0037] Fig. 9B It shows that Fig. 9A Schematic diagram of a single wafer example shown. DETAILED DESCRIPTION

[0038] Various aspects of the present disclosure provide a method for manufacturing a processing device machine with integrated sensing and machine learning capabilities. A processing device according to the present disclosure may include a sensing module and a machine learning (ML) processor integrated on the same semiconductor device using three-dimensional (3D) core integration or monolithic 3D integration. The sensing module may include a sensor array that can generate analog sensing data (e.g., an analog image signal generated by an image sensor). The ML processor may process the analog sensing data using one or more machine learning models.

[0039] In one embodiment, the ML processor may include a preprocessing unit that can preprocess the analog sensor data for ML processing, for example, by performing operations such as feature extraction, dimensionality reduction, image processing, etc. The preprocessing unit may include one or more crossbar arrays that can be used to preprocess the analog sensor data in the analog domain. Each crossbar array can be a circuit structure having interconnected conductive lines with a resistance conversion material sandwiched at the intersection. The resistance conversion material may include, for example, a memristor (also known as a resistance random access memory (RRAM or ReRAM)). The analog sensor data can be provided to the crossbar array as an input signal. The crossbar array can generate an analog output signal representing the preprocessed sensor data. Then, the analog output signal can be converted into a digital signal representing the preprocessed sensor data, and the ML processor performs subsequent machine learning processing. By preprocessing the analog sensing data in the analog domain and digitizing the preprocessed sensing data instead of the raw sensing data, the ML sensor described in the present disclosure can achieve significant data reduction because only a small amount of information (e.g., the preprocessed sensing data) can be digitized and transmitted from the sensing module at the edge to the next layer of the network.

[0040] In another embodiment, the ML processor can run a machine learning model on the analog sensor data and can generate an analog signal representing the predicted output of the ML processing (e.g., classification results, labels assigned to the analog sensor data, outputs of neural network layers, decisions made based on the ML model, etc.). For example, the ML processor can implement a multi-layer neural network using a crossbar array. The analog output signal generated by the crossbar array can represent the predicted output, can be converted to a digital output and can be transmitted to other computing devices.

[0041] In some embodiments, the processing device may be implemented by single-wafer integration. For example, the sensing module may be monolithically integrated with a circuit that implements an ML processor on a single substrate (e.g., a single crystal silicon substrate). The ML processor may include one or more CMOS circuits that include transistors, RRAM devices, interconnect layers, and / or any other suitable components for implementing an ML processor. The sensing module may include a photodiode, an interconnect layer, a color filter, a microlens, and / or any other suitable components for implementing a sensing module.

[0042] In one embodiment, the sensing module and the ML processor may be manufactured in non-overlapping areas on the substrate. For example, the sensing module may be manufactured on the central area of ​​the substrate. The ML processor may be manufactured on a second area of ​​the substrate, the second area surrounding the central area of ​​the substrate. The sensing module and the ML processor may be manufactured on the same side of the substrate. In another embodiment, the sensing module and the ML processor may be manufactured on different sides of the substrate, respectively. For example, a CMOS circuit implementing the ML processor may be manufactured on a first side of the substrate. The substrate is then flipped to expose the second side of the substrate (e.g., the reverse side of the substrate). The sensing module may be manufactured on the second side of the substrate. In another embodiment, a CMOS circuit implementing the ML processor may be manufactured on the substrate. The sensing module may then be manufactured on the CMOS circuit.

[0043] The monolithic integration of the sensing module and the ML processor can provide 3D heterogeneous integration of sensing and processing functions, and can realize the required hardware processing capabilities, such as near-sense processing, in-memory computing, analog computing, and parallel computing. The monolithic integrated sensing module and ML processor can be used to implement 3D neural network hardware, resulting in higher device density, complex connections, and reduced communication losses. In some embodiments, ML processing can also provide a two-dimensional interface (a cross-section of the 3D neural network) to communicate with a 2D sensor array (e.g., an image sensor array), which can realize that the sensing data generated by the 2D sensor array can directly enter the 3D neural network for processing without the need for signal storage or reconfiguration into one-dimensional data (e.g., a vector that can be used as an input to a traditional 2D neural network).

[0044] In addition, most sensor signals can be regarded as specific forms of energy (e.g., temperature, mechanical force, photons, vibrations, chemicals, electromagnetic waves, etc.), and can be easily converted into electrical signals (e.g., 0.5V) using new devices. These electrical signals are not only analog data to be processed, but can also be used as a potential energy source to self-power the sensor module and other components of the processing device. In some embodiments, the processing device is awakened only in the presence of a sensor signal, and event-driven applications can be implemented, further reducing the amount of data collected and the energy consumed for ML processing.

[0045] Figure 1A and 1B is a schematic diagram showing examples of processing devices 100a and 100b having integrated sensing and processing capabilities according to some embodiments of the present disclosure.

[0046] like Figure 1A As shown, the processing device 100 may include a sensing module 110, a machine learning (ML) processor 120, and a communication module 130. The ML processor 120 may further include a pre-processing module 121, an analog-to-digital conversion (ADC) unit 123, and a machine learning (ML) processing unit 125. The sensing module 110, the ML processor 120, and the communication module 130 may be integrated into a single chip, for example, as shown below. Fig. 7A -The single crystal described in 9.

[0047] The sensing module 110 may include one or more sensor arrays. Each sensor array may include one or more sensors that can detect and / or measure physical properties and generate electrical signals representing the physical properties. Examples of the sensors include image sensors, audio sensors, chemical sensors, pressure sensors, thermal sensors, temperature sensors, vibration sensors, microbial fuel cells, electromagnetic sensors, etc. In some embodiments, the multiple sensor arrays in the sensing module 110 may include different types of sensors. In some embodiments, the sensing module 110 may include a combination of Figure 5A-5B In some embodiments, the sensing module 110 may generate analog sensing data in the form of analog sensing signals (e.g., voltage signals, current signals, etc.), such as analog image signals generated by an array of image sensors (e.g., CMOS image sensors) that may detect light and generate analog image signals representing the detected light.

[0048] In some embodiments, the sensors in the sensing module 110 can collect enough energy to enable the ML processor 120 and / or the processing device 100a to operate without external power. For example, the electrical signal generated by the sensing module 110 can be used to power the ML processor 120 and / or the processing device 100a.

[0049] The ML processor 120 may process the analog sensor data generated by the sensor module 110 using one or more machine learning models. For example, the preprocessing unit 121 may process the analog sensor data and generate analog preprocessed sensor data. The preprocessing unit 121 may perform any suitable operation on the analog sensor signal to prepare the analog sensor data for subsequent operations of the ML processing unit 125. For example, the preprocessing unit 121 may perform feature extraction on the analog sensor data and extract features of the analog sensor data for subsequent ML processing. As another example, the preprocessing unit 121 may perform dimensionality reduction processing on the analog sensor data to reduce the amount of data to be processed in subsequent ML processing. As another example, the preprocessing unit 121 may perform one or more convolution operations (e.g., two-dimensional convolution operations, depth convolution operations, etc.) on the analog sensor data. As another example, the preprocessing unit 121 may normalize, scale and / or resize, denoise, etc. the analog sensor data. In some embodiments where the analog sensor data includes an analog image signal, the pre-processing unit 121 may apply a suitable image processing technique to process the analog sensor data.

[0050] The pre-processing unit 121 may include one or more crossbar arrays that can process analog sensor signals in the analog domain. Each crossbar array may include a plurality of interconnected conductive lines (e.g., row lines, column lines, etc.) and crosspoint devices fabricated at the intersections of the conductive lines. The crosspoint devices may include, for example, memristors, phase change memory devices, floating gates, spintronic devices, and / or any other suitable devices with programmable resistance. In some embodiments, the crossbar array may include, for example, a combination of Figure 2-3 The one or more crossbar arrays.

[0051] As an example, the cross array can receive an input voltage signal V and can generate an output current signal I. The relationship between the input voltage signal V and the output current signal I can be expressed as I=VG, where G represents the conductance value of the cross-point device. Therefore, the input signal of each cross-point device can be weighted by its conductance according to Ohm's law. The weighted current can be output through each bit line and can be accumulated according to Kirchhoff's current law. The conductance value of the cross-point device can be programmed to represent the values ​​and / or weights of one or more matrices used to perform preprocessing of the analog sensing signal as described above (e.g., feature extraction, dimensionality reduction, convolution, image processing, etc.). The cross array can receive as input an analog sensing signal generated by a sensor in the sensing module 110, and can generate an analog output signal (e.g., a current signal) representing the preprocessed analog sensing data.

[0052] In some embodiments, the sensing module 110 may include sensors arranged as a two-dimensional (2D) sensor array. Since each sensor can generate an analog sensing signal, the output of the two-dimensional sensor array can be regarded as a two-dimensional output including the analog sensing signal generated by the sensor (for example, m*n analog sensing signals generated by m*n sensors). The pre-processing unit 121 may include a three-dimensional (3D) cross array, which includes multiple two-dimensional cross arrays arranged in a three-dimensional manner. For example, the two-dimensional cross arrays can be located in different planes (for example, parallel planes perpendicular to the substrate on which the three-dimensional cross array is manufactured). The cross-section of the three-dimensional cross array is two-dimensional, and can receive and / or process the two-dimensional output (for example, m*n analog sensing signals) generated by the sensing module 110 without converting the two-dimensional output into one-dimensional data (for example, a vector representing the sensing signal generated by the sensor). The three-dimensional cross array circuit can be and / or include a combination of Figure 3 In some embodiments, the three-dimensional crossbar array circuit can be manufactured using the technology described in US Patent Application 16 / 521,975, entitled "Crossbar Array Circuit with 3D Vertical RRAM", which is incorporated by reference in its entirety into the present disclosure.

[0053] Analog-to-digital converter (ADC) 123 may include any suitable circuitry for converting analog pre-processed sensor data into digital pre-processed sensor data. In some embodiments, ADC 123 may include a combination of Figure 2 The one or more ADC250.

[0054] The ML processing unit 125 may include circuits for processing digital preprocessed sensor data using one or more machine learning models. In some embodiments, the ML processing unit 125 may include a digital signal processor. The ML processing unit 125 may obtain a predicted output by running a trained machine learning model using the digital preprocessed sensor data. The predicted output may represent, for example, a classification result (e.g., a category label assigned to the sensor data), a decision made based on a machine learning model, and the like. The machine learning model may refer to a model product created by a processing device using training data, the training data including known training inputs and corresponding known outputs (the correct answer for each training input). The processing device may find patterns in the training data that map known inputs to known outputs (predicted outputs), and provide a machine learning model that can obtain these patterns.

[0055] The machine learning model may include a machine learning model composed of a single-stage linear or nonlinear operation (e.g., a support vector machine), a neural network composed of a multi-stage nonlinear operation, etc. The neural network may include an input layer, one or more hidden layers, and an output layer. The neural network may be trained by, for example, adjusting the weights of the neural network according to a back propagation learning algorithm. In some embodiments, the crossbar array in the preprocessing unit 121 may implement one or more layers of the neural network. For example, the analog domain processed sensor data generated by the preprocessing unit 121 may represent the output of an input layer or a hidden layer in the neural network.

[0056] The communication module 130 may include any suitable hardware and / or software to facilitate communication between the processing unit 100a and one or more other computing devices. For example, the communication module 130 may include one or more transceivers that can transmit and / or receive RF (radio frequency) signals. The communication module 130 may include components for implementing one or more other wireless transmission protocols (e.g., Wi-Fi, Bluetooth, ZIGBEE, cellular light). In some embodiments, the communication module 130 may include one or more antennas that may be integrated in the Figure 4A-4B The communication module 130 may transmit the output of the ML processor 120 to other computing devices (e.g., cloud computing devices) for subsequent processing. In some embodiments, the communication module 130 may further receive instructions from the computing device for performing operations according to the predicted output (e.g., turning on a display based on the face recognition result, sending data to one or more other processing devices, presenting media content, etc.).

[0057] refer to Figure 1B, the processing device 100b may include a sensing module 110, a machine learning (ML) processor 140, and a communication module 130. The sensing module 110 and the communication module 130 may be combined with Figure 1A The corresponding parts described are the same.

[0058] The ML processor 140 may process the analog sensing data generated by the sensing module 110 using one or more machine learning models. The ML processor 140 may include a machine learning (ML) processing unit 141 and an ADC 143. The ML processing unit 141 may process the analog sensing data generated by the sensing module 110 using one or more machine learning models to generate a simulated prediction output. The simulated prediction output may include one or more analog signals.

[0059] In some embodiments, the ML processing unit 141 may include one or more crossbar arrays, each of which may include a combination of the following: Figure 2 In some embodiments, the ML processing unit 141 may include a combination of the following Figure 3The 3D cross array. In some embodiments, the cross array can implement a neural network that executes a machine learning algorithm. The output signal of the cross array can represent the output of the neural network. The neural network can include multiple convolution layers, each of which can perform a specific convolution operation (e.g., 2D convolution, deep convolution, etc.). Each layer of the neural network can be implemented using one or more cross arrays. For example, one or more first cross arrays can implement the first layer (e.g., input layer) of the neural network. The first cross array can receive the analog sensor data generated by the sensor module 110 as input and perform one or more convolution operations on the analog sensor data. Performing a convolution operation on the analog sensor data can involve convolving different parts of the sensor data using one or more kernels. For example, a 2D convolution can be performed by applying a single convolution kernel to the analog sensor data. More specifically, the convolution kernel can be used to scan each part of the sensor data that has the same size as the convolution kernel to produce a convolution result. As another example, performing a deep convolution on the sensor data can involve convolving each channel of the sensor data using a corresponding kernel and superimposing the convolution outputs together. For example, the conductance values ​​of the multiple crosspoint devices of the first cross array can be programmed to represent the value of the 2D convolution kernel. The analog sensor signal can be provided to the first cross array as an input signal. The first cross array can output a current signal representing the convolution of the analog sensor signal and the 2D convolution kernel. In some embodiments, the convolution array can store multiple 2D convolution kernels by mapping each 2D convolution kernel to the multiple crosspoint devices of the first cross array. The first cross array can output multiple output signals (e.g., current signals) representing the convolution results. The output of the first cross array can be provided to one or more second cross arrays that implement the second layer of the neural network for processing. The output of the second cross array (e.g., the analog current signal) can represent the output of the second layer of the neural network. The output of the second cross array can provide one or more cross arrays that implement the subsequent layer of the neural network (e.g., the second hidden layer) for processing. One or more third cross arrays can implement the output layer of the neural network. The output of the third cross array (e.g., the analog current signal) can represent the output of the neural network. In some embodiments, the neural network may be implemented using a crossbar array as disclosed in U.S. Patent Application 16 / 125,454, entitled "Implementing a Multilayer Neural Network Using a Crossbar Array," the entire contents of which are incorporated herein by reference.

[0060] ADC 143 may include any suitable circuit for converting the analog output of ML processing unit 141 into a digital output. The digital output may represent the predicted output. In some embodiments, ADC 143 may include Figure 2 ADC250 in.

[0061] The communication module 130 may transmit the output of the ML processor 140 to another computing device (e.g., a cloud computing device) for further processing. In some embodiments, the communication module 130 may further receive instructions from the computing device for performing operations based on the predicted output (e.g., turning on a display based on the face recognition result, transmitting data to other processing devices, presenting media content, etc.).

[0062] In some embodiments, the processing devices 100a-b may be self-powered and may operate without an external power source. For example, the sensing module 110 may power the components of the processing devices 100a-b. The ML processors 120 and 140 and their components may be powered by the analog outputs generated by the sensing module 110 to implement operations as described herein.

[0063] Figure 2 2 is a schematic diagram showing an example 200 of a cross array according to some embodiments of the present disclosure. As shown in the figure, the cross array 200 may include a plurality of interconnected conductive lines, for example, one or more row lines 211a, 211b, ..., 211i, ..., 211n and column lines 213a, 213b, ..., 213j, ..., 213m, for an n-row by m-column cross array. The cross array 200 may further include cross point devices 220a, 220b, ..., 220z, etc. Each cross point device may connect a row line and a column line. For example, the cross point device 220ij may connect the row line 211i and the column line 213j. The number of column lines 213a-m and the number of row lines 211a-n may be the same or different. The cross array 200 may also include a word line (WL) logic 205 connected to the cross point device via the row lines 211a-n. The WL logic 205 may include any suitable components for applying input signals to the selected cross-point devices via the row lines 211a-n, such as one or more digital-to-analog converters (DACs), amplifiers, etc. Each input signal may be a voltage signal, a current signal, etc. The input signal may correspond to Figure 1A-1B The analog sensing signal generated by the sensing module 110.

[0064] The row lines 211a-n may include a first row line 211a, a second row line 211b, ..., 211i, ..., and an nth row line 211n. Each of the row lines 211a, ..., 211n may be and / or include any suitable conductive material. In some embodiments, each row line 211a-n may be a metal line.

[0065] The column lines 213 may include a first column line 213a, a second column line 213b, ..., and an mth column line 213m. Each column line 213a-m may be and / or include any suitable conductive material. In some embodiments, each column line 213a-m may be a metal line.

[0066] Each cross-point device 220 may be and / or include any suitable device having adjustable resistance, such as a memristor, a phase change memory (PCM) device, a floating gate, a spintronic device, a ferroelectric device, a RRAM device, or the like.

[0067] Each row line 211a-n can be connected to one or more row switches 231 (e.g., row switches 231a-231n). Each row switch 231 can include any suitable circuit structure that can control the current flowing through the row line 211a-211n. For example, the row switch 231 can be a CMOS switch circuit, or include a CMOS switch circuit.

[0068] Each column line 231a-m can be connected to one or more column switches 233 (e.g., switches 233a-233m). Each column switch 233a-233m can include any suitable circuit structure that can control the current flowing through the column line 213a-213m. For example, the column switches 233a-233m can be and / or include CMOS switch circuits. In some embodiments, one or more of the switches 231a-231n and 233a-233m can further provide fault protection, electrostatic discharge (ESD) protection, noise reduction, and / or any other suitable function for one or more portions of the crossbar array 200.

[0069] The output sensor 240 may include any suitable components that convert the current flowing through the column lines 213a-n into an output signal, such as one or more TIAs (transimpedance amplifiers) 240a-n. Each TIA 240a-n can convert the current flowing through the corresponding column line into a corresponding voltage signal. Each ADC 250a-n can convert the voltage signal generated by the corresponding TIA into a digital output. In some embodiments, the output sensor 240 may further include one or more multiplexers (not shown).

[0070] The programming circuit 260 can program the cross-point device 220 selected by the switch 231 and / or 233 to a suitable conductance value. For example, programming the cross-point device can involve applying a suitable voltage signal or current signal to the cross-point device. The resistance of each cross-point device can be electrically switched between a high-resistance state and a low-resistance state. Setting the cross-point device can involve switching the resistance of the cross-point device from a high-resistance state to a low-resistance state. Resetting the cross-point device can involve switching the resistance of the cross-point device from a low-resistance state to a high-resistance state.

[0071] The crossbar array 200 can perform parallel weighted voltage multiplication and current summation. For example, an input voltage signal can be applied to one or more rows (e.g., one or more selected rows) of the crossbar array 200. The input signal can flow through the crosspoint devices of the rows of the crossbar array 200. The conductance of the crosspoint device can be adjusted to a specific value (also referred to as a "weighted value"). According to Ohm's law, the input voltage is multiplied by the crosspoint conductance and generates a current flowing through the crosspoint device. According to Kirchhoff's law, the sum of the currents through the devices on each column generates a current as an output signal, which can be read from the column (e.g., the output of an ADC). According to Ohm's law and Kirchhoff's current law, the input-output relationship of the crossbar array can be expressed as I=VG, where I is the output signal matrix, expressed as current; V is the input signal matrix, expressed as voltage; and G is the conductance matrix of the crosspoint device. Therefore, the input signal is weighted by its conductance at each crosspoint device according to Ohm's law. The weighted current is output through each column line and accumulated according to Kirchhoff's current law. This can be achieved by implementing parallel multiplications and sums in a crossbar array to achieve in-memory computing (IMC).

[0072] The crossbar array 200 may be configured to perform vector matrix multiplication (VMM). A VMM operation may be represented as Y=XA, where Y, X, and A represent matrices, respectively. More specifically, for example, the input vector X may be mapped to an input voltage V of the crossbar array 200. The matrix A may be mapped to a conductance value G. The output current I may be read and mapped back to an output result Y. In some embodiments, the crossbar array 200 may be configured to implement a portion of a neural network by executing a VMM.

[0073] In some embodiments, the cross array 200 can perform a convolution operation. For example, performing a 2D convolution on input data can involve applying a single convolution kernel to the input signal. Performing a deep convolution on the input signal can involve convolving each channel of the input data with a corresponding kernel corresponding to the channel and stacking the convolution outputs together. The convolution kernel can have a specific size defined by multiple dimensions (e.g., width, height, channel, etc.). The convolution kernel can be applied to a portion of the input data having the same size as the kernel to produce an output. The output can be mapped to an element of the convolution result that is located at a position corresponding to the position of the input data portion.

[0074] The programming circuit 260 can program the cross array 200 to store a convolution kernel for performing a 2D convolution operation. For example, the convolution kernel can be converted into a vector and mapped to a plurality of cross-point devices in the cross array connected to a specified bit line. Specifically, the conductance value of the cross-point device can be programmed to represent the value of the convolution kernel. In response to an input signal, the cross array 200 can output a current signal representing the convolution of the input signal and the 2D convolution kernel through a specified bit line. In some embodiments, the cross array 200 can store multiple 2D convolution kernels by mapping each 2D convolution kernel to a cross-point device connected to a corresponding bit line. The cross array 200 can output multiple output signals (e.g., current signals) representing the convolution results through the column line 213.

[0075] Figure 3 FIG. 4 is a schematic diagram showing an example 3D crossbar array circuit 300 according to some embodiments of the present disclosure.

[0076] As shown in the figure, the 3D cross array circuit 300 may include a first cross array 310, a second cross array 320, and a third cross array 330 located in different planes. In some embodiments, the first cross array 310, the second cross array 320, and the third cross array 330 may be located on a first plane, a second plane, and a third plane, respectively. The first plane, the second plane, and the third plane are parallel to each other. In some embodiments, the first plane, the second plane, and the third plane may be perpendicular to or parallel to the substrate formed by the first cross array 310, the second cross array 320, and the third cross array 330. Each of the first cross array 310, the second cross array 320, and the third cross array 330 may include one or more combinations of Figure 2 The 2D crossbar array. Figure 3 While three crossbar arrays are shown, the 3D crossbar array circuit 300 may include any suitable number of 2D crossbar arrays integrated into a 3D crossbar array circuit.

[0077] The first crossbar array 310 may include a crosspoint device 315 connecting a first plurality of word lines (WL1_1, WL2_1, WL3_1, etc.) and a first plurality of bit lines (BL1_1, BL2_1, BL3_1, etc.). The second crossbar array 320 may include a crosspoint device 325 connecting a second plurality of word lines (WL1_2, WL2_2, WL3_2, etc.) and a second plurality of bit lines (BL1_2, BL2_2, BL3_2, etc.). The third crossbar array 330 may include a crosspoint device 335 connecting a third plurality of word lines (WL1_3, WL2_3, WL3_3, etc.) and a third plurality of bit lines (BL1_3, BL2_3, BL3_3, etc.).

[0078] The 3D crossbar array circuit 300 may further include transistors 340. Each transistor 340 may be connected to a corresponding gate line (e.g., GL1, GL2, GL3, etc.) through its gate region. For example, the gate line GL1 may be connected to the gate region of the first transistor in the first crossbar array 310, the gate region of the second transistor in the second crossbar array 320, and the gate region of the third transistor in the third crossbar array 330. The source region of the corresponding transistor 340 may be connected to a word line. It should be noted that Figure 3 The components of the 3D crossbar array circuit 300 and their connections are schematically shown. Figure 3 The schematic diagram shown does not represent the physical layout of the components of the 3D crossbar array circuit 300. The components of the 3D crossbar array circuit 300 can be physically laid out and positioned in any suitable manner to implement the 3D crossbar array shown in the present disclosure. For example, in one physical layout of the 3D crossbar array circuit 300, the transistors 340 can be located in the same layer of the substrate and then connected to the corresponding gate lines, bit lines, and word lines through vertical vias.

[0079] To select the cross-point device located at the intersection of WL3_3 and BL3_3, a voltage VG can be applied to GL3, while the other GLs can be grounded, so that the transistor channel on GL3 is turned on. A voltage VD can be applied to the drain region of the transistor connected to WL3_3, while the drain regions of other transistors located at the same horizontal layer are grounded, so that current can only pass through WL3_3. A voltage Vground can be applied to BL3_3, while a voltage VS can be maintained on other BLs that intersect WL3_3. VS can be equal to Vd-Vds, where VS represents the voltage of the source region of the transistor and Vds represents the voltage drop between the drain region and the source region of the transistor. Therefore, only one device on the WL can be programmed by the voltage difference between VS and Vground. Other devices on the same WL (WL3_3) cannot be programmed because there is no voltage difference on these devices.

[0080] Since the crossbar arrays are arranged in a 3D manner in the 3D crossbar array circuit 300, the cross section of the 3D crossbar array circuit 300 can be regarded as a 2D crossbar array. Therefore, the 3D crossbar array circuit 300 can receive and process 2D inputs (e.g., m*n analog input signals) without storing the 2D inputs or converting the 2D inputs into one-dimensional data (e.g., vectors representing the 2D inputs). For example, as described above, the crosspoint devices located on WL1_1, WL1_2, WL1_3, etc. can be selected to receive and process the analog sensing signals generated by the 2D sensor array.

[0081] Figure 4A and 4BSchematic diagrams showing examples of semiconductor devices 400a and 400b that can be used as machine learning processors in some embodiments according to the present disclosure.

[0082] like Figure 4A As shown, semiconductor device 400a may include a single wafer 420 and a package substrate 410. Package substrate 410 may include an antenna, a connector, a power supply, etc. In some embodiments, package substrate 410 does not include a CMOS component. Single wafer 420 may be connected to package substrate 410 via an interconnect layer 430. For example, interconnect layer 430 may include ball grid array (BGA) bumps. In some embodiments, package substrate 410 may be connected to a PCB (printed circuit board) substrate (not shown).

[0083] The single chip 420 may include one or more sensor arrays consisting of a sensor module (e.g., a combination of Figure 1A-1B The sensor module 110 described above). In some embodiments, the single wafer 420 may include a Figure 5A-5B The one or more image sensor wafers.

[0084] The single wafer 420 may further include an ADC, a crossbar array, a driver IC (integrated circuit), a CMOS component for implementing a transceiver, and any other suitable component for implementing machine learning processing. In some embodiments, the single wafer 420 may include a chip for implementing Figure 1A ML processor 120 and / or Figure 1B One or more CMOS circuits of the ML processor 140 (e.g., in combination with Figure 6A-6B The CMOS circuit 600a-b described above). The single chip 420 may include the following: Fig. 7A -The single crystal wafer as described in 9.

[0085] refer to Figure 4B In some embodiments, a plurality of single wafers may be stacked on the package substrate 410. As shown, the single wafers 420a, 420b, and 420c may be connected to the package substrate 410 through interconnection layers 430a, 430b, and 430c, respectively. In some embodiments, the single wafers 420a, 420b, and 420c may include different types of sensors, may sense different signals, and may process different sensing signals generated by the sensors. Each of the single wafers 420a-c and the interconnection layers 430a-c may be and / or include a combination of Figure 4A Its counterparts (eg, single wafer 420 and interconnect layer 430) are described above. Although Figure 4A and 4B A specific number of wafers is shown in FIG. 4 , but this is merely illustrative. Any suitable number of single wafers may be stacked on the packaging substrate 410 as described herein.

[0086] Figure 5A and 5B is a schematic diagram showing a cross-sectional view of example image sensor wafers 500a and 500b according to some embodiments of the present disclosure. Each of the image sensor wafers 500a and 500b may be considered a CMOS image sensor (CIS) wafer.

[0087] like Figure 5A As shown, image sensor wafer 500a may include an image sensor having microlenses 511, color filters 513, and photodiodes 515a. Photodiodes 515a may be fabricated on substrate 505 (eg, a silicon substrate). Substrate 505 may be and / or include Figures 7A-7F The substrate 710 and / or Fig. 8A -9. The metal line 520a may be located between the color filter 513 and the photodiode 515a.

[0088] Incident light may be focused by a microlens 511 and may be separated into a plurality of color components by one or more sets of color filters 513. For example, a red filter 513a, a green filter 513b, and a blue filter 513c may separate a red component, a green component, and a blue component, respectively. A photodiode 515a may accumulate photon charges when exposed to light and may convert the charges into electrical signals (voltage signals).

[0089] refer to Figure 5B , the image sensor wafer 500b includes a back-illuminated structure, in which the photodiode 515b is arranged behind the color filter 513, and the metal line 520b is arranged behind the photodiode 515b. The image sensor wafer 500b can be manufactured by manufacturing the photodiode 515b and the metal line 520b on the front silicon substrate 505, then flipping the substrate 505, thinning the back substrate, and manufacturing the color filter 513 and the microlens 511 on the back. As shown in the figure, the metal line 520b is located behind the photodiode 515b, and the metal line 520a is located in front of the photodiode 515a. Light can be incident on the interface 531a of the image sensor wafer 500a and the interface 531b of the image sensor wafer 500b respectively. Therefore, since the light can reach the photodiode 515b without passing through the metal line 520b, the photodiode 515b can obtain more light signals than the photodiode 515a.

[0090] Fig. 6A and 6B 6 is a schematic diagram showing cross-sectional views of example CMOS circuits 600a and 600b including CMOS-compatible RRAMs according to some embodiments of the present disclosure.

[0091] As shown in the figure, transistor 603 is fabricated on substrate 601. The transistor 603 may include a source region 603a, a gate 603b and a drain region 603c. Fig. 6A A transistor is shown in the figure, but it is illustrative only. In some embodiments, multiple transistors (not shown) can be manufactured on substrate 601. The transistors can be isolated by suitable insulators and / or dielectric materials. Substrate 601 can include any suitable material that can be used as a substrate for making CMOS circuits, such as silicon (Si), silicon dioxide (SiO2), silicon nitride (Si3N4), aluminum oxide (Al2O3), aluminum nitride (AlN), etc.

[0092] CMOS circuit 600a may include interconnect layers 610 fabricated on transistor 603 and substrate 601. Each of interconnect layers 610 may provide electrical connections between transistor 603 and / or one or more other devices (e.g., one or more other transistors, one or more RRAM devices, etc.). Interconnect layers 610 may include, for example, via layers 611, 612, 613, 614, ..., and 615 and metal layers 621, 622, 623, 624, ..., and 625. Although for simplicity of illustration, Figure 6A-6B 615 via layers and up to 625 pad layers are shown, but additional via layers and pad layers may be manufactured for integration and / or interconnection needs. Each via layer may include one or more metal vias. Each metal via may include a suitable metal material, such as Al, Cu, W, etc. Each metal layer may include one or more metal pads. Each metal pad may include a suitable metal material, such as Al, Cu, W, etc. For example, the via layer 611 may include metal vias 611a, 611b, and 611c connected to the source region 603a, the gate 603b, and the drain region 603c, respectively. In some embodiments, the via layer 611 may include a tungsten (W) via and a doped polysilicon (poly-Si) terminal, wherein the poly-Si terminal may directly contact the gate 603b, the source region 603a, and the drain region 603c of the transistor 603. The tungsten via may directly contact the poly-Si terminal. Other via layers and metal layers on the via layer 611 may be made of Cu, W, Al, etc. The metal layer 621 may include metal pads 621a, 621b, and 621c. The metal pads 621a, 621b, and 621c may be connected to the metal vias 611a, 611b, and 611c, respectively.

[0093] Each interconnect layer can be manufactured by manufacturing a dielectric layer, patterning the dielectric layer, and depositing a suitable metal on the patterned dielectric layer. The dielectric layer may include any suitable dielectric material, such as silicon nitride (Si3N4), silicon dioxide (SiO2), etc. For example, in order to manufacture the first via layer 611, the dielectric layer 651 can be manufactured on the substrate 601 and the transistor 603. The dielectric layer 651 can be processed using any suitable deposition technology. For example, the dielectric layer 651 can be patterned and filled by metal deposition to manufacture metal vias 611a, 611b, and 611c in the dielectric layer 651. In some embodiments, one or more interconnect layers 610 can be manufactured using a dual metal damascene process, wherein the metal vias of the via layer and the metal pads of the metal layer can be deposited and patterned using the same metal process.

[0094] As shown in the figure, a pair of adjacent metal layers can be connected by a via layer made between the adjacent metal layers. For example, the first metal layer 621 can be connected to the second metal layer 622 through the via layer 612. More specifically, the metal pad 622a of the metal layer 622 can be connected to the metal pad 621a of the metal layer 621 through the metal via 612a. The metal pad 622b of the metal layer 622 can be connected to the metal pad 621b of the metal layer 621 through the metal via 612b. Figure 6B As shown, metal pad 622 c of metal layer 622 may be connected to metal pad 621 c of metal layer 621 through metal via 612 c.

[0095] The interconnect layer 610 may have different sizes. The sizes of the metal pads of the metal layers 621, 622, 623, 624, ..., and 625 may increase in sequence. Similarly, the sizes of the metal vias of the via layers 611, 612, 613, 614, ..., and 615 may also increase in sequence. For example, the CMOS circuits 600a-b may be of a 65nm process node. The width and spacing between the metal pads of the metal layer 621 may be about 90nm. The width and spacing of the metal pads in the metal layers 622 and 623 may be about 100nm. The width and spacing of the metal pads in the metal layer 625 may be about 400nm.

[0096] The RRAM device 640 can be manufactured during the interconnect layer manufacturing process. Therefore, the RRAM device 640 can be regarded as a CMOS compatible RRAM device. For example, one or more first interconnect layers 610a can be manufactured on the transistor 603 and / or the substrate 601. The RRAM device 640 can be manufactured on a metal pad or a metal via on the top interconnect layer of the first interconnect layer 610a. Then one or more second interconnect layers 610b can be manufactured on the RRAM device 640 and the first interconnect layer 610a. More specifically, for example, a metal pad or a metal via in the bottom interconnect layer of the second interconnect layer 610b can be manufactured on the RRAM device 640 and can directly contact the RRAM device 640. In some embodiments, as Fig. 6A As shown, the first interconnect layer 610a may include a via layer 611 and a metal layer 621. The metal layer 621 may be considered as a top interconnect layer of the first interconnect layer 610a. The RRAM device 640 may be fabricated on a metal pad 621c of the metal layer 621. The RRAM device 640 may be connected to a drain region 603c of the transistor 603 through the metal pad 621c of the metal layer 621 and the metal via 611c of the via layer 611. The metal via 612c of the via layer 612 may be fabricated on the RRAM device 640 and may be connected to a bit line (e.g., Figure 2 The metal vias 612a and 612b of the via layer 612 may be fabricated on metal pads 621a and 621b, respectively. The metal layer 621 may be considered as a bottom interconnect layer of the second interconnect layer 610b. The second interconnect layer 610b may include one or more metal layers and / or via layers (e.g., metal layers 622, 623, 624, and 625, and via layers 613, 614, and 615) fabricated on the metal layer 621.

[0097] In some embodiments, Figure 6B As shown, the RRAM device 640 can be manufactured on the metal pad 622c of the metal layer 622. The via layer 613 can be manufactured on the RRAM device 640. More specifically, the metal via 613c of the via layer 613 is manufactured on the RRAM device 640 and directly contacts the RRAM device 640. In this embodiment, the first interconnect layer 610a may include a via layer 611, a metal layer 621, a via layer 612, and a metal layer 622. The metal layer 622 can be regarded as the top interconnect layer of the first interconnect layer 610a. The second interconnect layer 610b may include via layers 613, 614, and 615 and metal layers 623, 624, and 625. The via layer 613 can be regarded as the bottom interconnect layer of the second interconnect layer 610b.

[0098] although Fig. 6A and Figure 6B The overall steps for making the interconnect layer 610 are the same, but Figure 6B The first interconnect layer 610a is fabricated by Fig. 6A There are more steps in manufacturing the first interconnect layer 610a, while Figure 6B The second interconnect layer 610b is fabricated by Fig. 6A The RRAM device 640 is connected to the drain region 603c of the transistor 603 through the metal pad 622c of the metal layer 622, the metal via 612c of the via layer 612, the metal pad 621c of the metal layer 621, and the metal via 611c of the via layer 611. The metal via 613c of the via layer 613 can be fabricated on the RRAM device 640 and can be connected to the bit line (e.g., Figure 2 The metal vias 613a and 613b of the via layer 613 may be fabricated on the metal pads 622a and 622b, respectively.

[0099] Although Figure 6A-6B Specific interconnect layers (e.g., metal layers and via layers) are shown, which is illustrative only. The CMOS circuits 600a-b may include any suitable number of interconnect layers for implementing various integrated circuits. The first interconnect layer 610a and the second interconnect layer 610b may include any suitable number of interconnect layers. For example, in some embodiments the RRAM device 640 may be fabricated on the metal layer 623 or other metal layers. In some embodiments, the RRAM device 640 may be connected to the source region 603a of the transistor 603. In this embodiment, the RRAM device 640 may be fabricated on a metal pad and / or metal via connected to the source region 603a.

[0100] Fig. 7A and 7B 700a and a top view 700b of a single wafer with integrated sensing and ML processing capabilities according to some embodiments of the present disclosure. The single wafer may be and / or include a combination of Figure 4A-4B The single crystal.

[0101] refer to Fig. 7A, a single wafer may include a sensing module 720 and an ML processor 730 fabricated on a substrate 710. The substrate 710 may be and / or include any suitable material suitable for fabricating the sensing module and components of the ML sensor described in the present disclosure. For example, the substrate 710 may include a silicon substrate or any other suitable material for fabricating the CMOS circuit and sensing module described in the present disclosure. The sensing module 720 and the ML processor 730 may be fabricated on the same side of the substrate 710 and on non-overlapping portions of the two substrates 710. For example, Figure 7B As shown, the sensing module 720 may be manufactured on a first portion (e.g., a central portion) of the substrate 710. The ML processor 730 may be manufactured on a second portion of the substrate 710 surrounding the first portion. This arrangement of the sensing module 720 and the ML processor 730 may reduce the wiring distance of the sensing module 720 and the ML processor 730. It may be noted that the sensing module 720 and the ML processor 730 may be arranged side by side on the substrate 710 in any other suitable manner.

[0102] The sensing module 720 may include a plurality of sensors (eg, a sensing array), interconnect layers (eg, metal wiring), and / or Figure 1A-1B any other suitable components of the sensor module 110 in the embodiment. For example, the sensor module 720 may include a plurality of photodiodes, color filters, and micro lenses for implementing an image sensor array. In some embodiments, the sensor module 720 may include a combination of Figure 5A-5B The CIS wafer 500a and / or 500b. The ML processor 730 may include transistors, RRAM devices, interconnect layers and / or Figure 1A-1B any other suitable components of the ML processors 120 and / or 140 in the ML processors 120 and / or 140. In some embodiments, the ML processor 730 may include one or more crossbar arrays (e.g., in combination with the ML processors 120 and / or 140) for implementing various functional components of the ML processors 120 and / or 140. Figure 2-3 Each cross array may include: Figure 6A-6B The one or more CMOS circuits 600a-b.

[0103] Figure 7C-7I It shows the combination Figure 7A-7B Schematic diagrams of cross-sectional views of structures 700c, 700d, 700e, 700f, 700g, 700h and 700i for manufacturing a single wafer.

[0104] like Figure 7CAs shown, transistors 731 and 733 and photodiodes 721a, 721b, ..., 721n of the ML processor 730 can be manufactured on a substrate 710 (e.g., a surface 711 of the substrate 710). Each photodiode 721a-721n can be a two-terminal device including a pn junction, which can convert photons into current. For example, each photodiode 721a-721n can include a p-doped region and an n-doped region (e.g., a p-doped region 7211a and an n-doped region 7213a of the photodiode 721a, a p-doped region 7211b and an n-doped region 7213b of the photodiode 721b, a p-doped region 7211n and an n-doped region 7213n of the photodiode 721n, etc.). In some embodiments, one or more photodiodes 721a-721n can be a pin structure, where i represents an undoped intrinsic region (not shown) between the p-doped region and the n-doped region. When a bias is applied to the photodiode, the current output can be controlled to provide a threshold, linear response, or nonlinear response. Specifically, the photodiodes 721a-721n can be placed in a reverse bias or a small forward bias so that the output current is a linear function of the input light intensity. Transistor 731 may include a source region 731a, a gate region 731b, and a drain region 731c. Transistor 733 may include a source region 733a, a gate region 733b, and a drain region 733c. Although Figure 7C A specific number of transistors and photodiodes are shown in FIG. 7 , but this is illustrative only. ML processor 730 may include any suitable number of transistors fabricated on substrate 710. Sensing module 720 may include any suitable number of photodiodes fabricated on substrate 710. The transistors may be fabricated on substrate 710 in an area surrounding the photodiodes.

[0105] like Fig.7D As shown, one or more interconnect layers 750a (also referred to as "first interconnect layers") may be fabricated on the transistors of the ML processor 730 and the photodiodes of the sensor module 720. The first interconnect layer 750a may be and / or include a combination of Figure 6A-6B The metal layer and / or through-hole layer. Fig.7DAs shown, the first via layer 741 can be manufactured on the transistor 731, the transistor 733 and the photodiodes 721a-n. The first via layer 741 can include a plurality of metal vias 741a, 741b, 741c, ..., 741i. The metal vias 741a, 741b and 741c can be manufactured on the source region 731a, the gate region 731b and the drain region 731c of the transistor 731, respectively. The metal vias 741d and 741e can be manufactured on the photodiode 721a. The metal vias 741f and 741g can be manufactured on the photodiode 721b. The metal vias 741h and 741i can be manufactured on the photodiode 721n. The metal vias 741j, 741k and 741i can be manufactured on the source region 733a, the gate region 733b and the drain region 733c of the transistor 733, respectively. The first metal layer 751 can be manufactured on the first via layer 741. The first metal layer 751 can include metal pads 751a-741i respectively manufactured on the metal vias 741a-741i. In some embodiments, the metal pads of the first metal layer 751 can directly contact the metal vias of the first metal via layer 741. Each interconnect layer 750a can be manufactured by manufacturing a dielectric layer (e.g., dielectric layers 761 and 763), patterning the dielectric layer, and depositing a suitable metal on the patterned dielectric layer. In some embodiments, the via layer 741 and the metal layer 751 can be manufactured using a dual metal damascene manufacturing process.

[0106] One or more RRAM devices may be fabricated on the top interconnect layer of the first interconnect layer 750a. For example, RRAM devices 735 and 737 may be fabricated on metal pads 751c and 751h, respectively. It should be noted that RRAM devices 735 and 737 may be fabricated on any suitable interconnect layer as described in the present disclosure. RRAM devices 735 and 737 may or may not be fabricated on the same interconnect layer.

[0107] like Fig. 7E As shown, one or more interconnect layers 750b (also referred to as "second interconnect layers") may be fabricated on the RRAM devices 735 and 737. The interconnect layers 750a and 750b may be referred to as interconnect layers 750. Each second interconnect layer 750b may be a combination of Figure 6A-6B The metal layer and / or via layer. For example, the second interconnect layer 750b may include via layers 743, ..., 745, and metal layers 753, ..., 755. The metal pads 755a-755i of the metal layer 755 may be connected to the metal pads 753a-753i of the metal layer 753 through the metal vias 745a-745i ​​of the via layer 745, respectively. The metal pads of the metal layers 753 and 751 may be connected through the metal vias of the via layer 743. Although Fig. 7E, but this is illustrative only. The second interconnect layer 750b can include any suitable number of interconnect layers. Each second interconnect layer 750b can be manufactured by manufacturing a dielectric layer (e.g., dielectric layers 765, 767, 769, and 771), patterning the dielectric layer, and depositing a suitable metal on the patterned dielectric layer. In some embodiments, the via layer and the metal layer (e.g., via layer 743 and metal layer 753) can be manufactured using a dual metal damascene process. In some embodiments, one or more through silicon vias (TSV) connections (not shown) can be manufactured together with the interconnect layer 750 to provide a connection between a single wafer and a packaging substrate.

[0108] like Figure 7F As shown, color filters 723a, 723b, ..., and 723n can be manufactured on the second interconnect layer 750b and / or the interconnect layer 750. In some embodiments, a set of color filters 723a-n can separate the incident light into multiple color components. For example, color filters 723a, 723b, and 723n can separate the red component, green component, and blue component of the incident light, respectively. Micro lenses 725a, 725b, ..., 725n can be manufactured on the color filters 723a-n, respectively. Although Figure 7F Two pairs of microlenses and color filters are shown, but this is illustrative only. Any suitable number of color filters and microlenses can be fabricated on interconnect layer 750.

[0109] In some embodiments, the Fig. 7E The semiconductor structure 700e in the manufacturing process is combined with Figure 5B The back-illuminated structure. Figure 7G As shown, semiconductor structure 700e may be flipped to include a back side (eg, surface 713) of substrate 710. Figure 7H As shown, the substrate 710 can be thinned to a suitable thickness so that incident light can reach the photodiode. The thinned back substrate can also be regarded as substrate 710a. Fig.7I As shown, color filters 723a-n and microlenses 725a-n can be fabricated on surface 713a of substrate 710a. Surfaces 713a and 711 correspond to two opposite sides of substrate 710.

[0110] Figures 8A-8B 8 are schematic diagrams showing cross-sectional views 800a and 800b of an example of a single wafer having integrated sensing and processing capabilities according to some embodiments of the present disclosure.

[0111] refer to Fig. 8A, the ML processor 820 can be manufactured on a substrate 810. The substrate 810 can be a single crystal silicon substrate in some embodiments. The substrate 810 can then be flipped over, and the sensor module 830 can be manufactured on the second side of the substrate 810. Therefore, the ML processor 820 and the sensor module 830 can be manufactured on different sides (e.g., opposite sides) of the substrate 810 using double-sided wafer processing and / or double-sided wafer processing technology. The substrate 810 can be a double-sided polished semiconductor wafer (e.g., a silicon wafer) having two polished surfaces and / or polished sides that can be used to manufacture the CMOS circuits and sensor modules described in the present disclosure. For example, a plurality of CMOS circuits (e.g., Figure 6A-6B The CMOS circuits 600a and / or 600b in the embodiment may be fabricated on the first side of the substrate 810 to realize the crossbar array and Figure 1A-1B The sensing module 830 may be manufactured in a Figure 8B The second side of the substrate 810 shown. The sensing module 830 may include one or more sensors fabricated on the substrate 810. In some embodiments, the sensing module 830 may include a photodiode, a color filter, one or more interconnect layers, a microlens, and any other suitable components for implementing the image sensor described in the present disclosure. For example, the sensing module 830 may include a combination of Figure 5A-5B The one or more sensor wafers 500a-b.

[0112] Figure 8C , 8D 8E and 8F are diagrams showing the combination of some embodiments of the present disclosure. Figures 8A-8B Schematic diagrams of cross-sections of example structures 800c, 800d, 800e, and 800f of the fabricated single wafer.

[0113] To manufacture a single wafer, the transistors of the ML processor 820 and the photodiodes of the sensor module 830 may be first manufactured on the substrate 810, because the manufacture of the transistors and the photodiodes may involve high temperature processing (e.g., thermal oxidation, ion implantation, diffusion, etc.). Figure 8C As shown, transistor 821 may be fabricated on substrate 810. Transistor 821 may include a source region 821a, a gate region 821b, and a drain region 821c. Substrate 810 may be flipped to expose a second side (eg, back side) of substrate 810. Fig.8D As shown, one or more photodiodes 831a-831n may then be fabricated on the second side of substrate 810. Figure 8C-8D A specific number of transistors and photodiodes are shown, but this is illustrative only. Any suitable number of transistors and photodiodes may be fabricated on substrate 810 to implement the ML processor described in the present disclosure.

[0114] refer to Fig. 8E One or more interconnect layers (also referred to as first plurality of interconnect layers) may be fabricated on transistor 821. The first plurality of interconnect layers may include metal layers 851, 853, ..., 855 and via layers 841, 843, ..., 845. Each first plurality of interconnect layers may be a combination of Figure 6A-6B The interconnection layer. The via layer 841 may include metal vias 841a, 841b, and 841c respectively connected to the source region 821a, the gate region 821b, and the drain region 821c of the transistor 821. The metal layer 851 may include metal pads 851a, 851b, and 851c respectively connected to the metal vias 841a, 841b, and 841c. The first plurality of interconnection layers may include any suitable number of metal layers and via layers. The RRAM device 823 may be manufactured during the manufacturing process of the interconnection layer. For example, the RRAM device 823 may be manufactured on the metal pad 851c of the metal layer 851. The metal via 843c of the via layer 843 may be manufactured on the RRAM device 823. The metal vias 843a and 843b in the via layer 843 may be connected to the metal pads 851a and 851b in the metal layer 851, respectively. In some embodiments, the RRAM device 823 may be fabricated on a metal pad and / or a metal via connected to the source region of the transistor 821 .

[0115] refer to Figure 8F, a plurality of interconnect layers (also referred to as "second plurality of interconnect layers") may be fabricated on the photodiodes 831a-n. The second plurality of interconnect layers may include, for example, via layers 861, 863, ..., 865 and metal layers 871, 873, ..., 875. The via layer 861 may include metal vias 861a, 861b, ..., 861n fabricated on the photodiodes 831a-n. The metal layer 871 may include metal pads 871a, 871b, ..., 871n fabricated on the metal vias 861a, 861b, ..., 861n, respectively. The metal pads 875a, 875b, ..., 875n may be connected to the metal pads 873a, 873b, ..., 873n of the metal layer 873 through the metal vias 865a, 865b, ..., 865n of the via layer 865. Metal pads 873a, 873b, ..., 873n may be connected to metal pads 871a, 871b, ..., 871n of metal layer 871 through metal vias 863a, 863b, ..., 863n of via layer 863. The second plurality of interconnect layers may include any suitable number of metal layers and via layers. Color filters 833a-n may be fabricated on the second plurality of interconnect layers (e.g., metal pads 875a, 875b, ..., and 875n in metal layer 875). Microlenses 835a-n may be fabricated on color filters 833a-n, respectively. In some embodiments, through silicon vias (not shown) may be fabricated to connect ML processor 820 to sensing module 830. In some embodiments, sensing module may be fabricated on ML processor 820.

[0116] Fig. 9A is a schematic diagram showing a cross-sectional view of a single wafer example 900a having integrated sensing and processing capabilities according to some embodiments of the present disclosure. Fig. 9B It shows Fig. 9A Schematic diagram of a single wafer example shown.

[0117] like Figures 9A-9B As shown, the ML processor 820 can be manufactured in combination with Fig. 8AThe substrate 810 is described. The sensing module 930 can be manufactured on the ML processor 820. The sensing module 930 may include one or more photodiodes manufactured on the interconnect layer in the ML processor 820, such as one or more amorphous silicon (a-Si) photodiodes, polycrystalline silicon (poli-Si) photodiodes and / or any other suitable photodiodes and / or sensors that can be manufactured on the CMOS circuit as described in the present disclosure. The sensing module 930 may further include a color filter, a microlens and / or any other suitable component manufactured on the photodiode. Therefore, since the sensing module can be stacked on the ML processor by depositing a-Si photodiodes, poli-Si photodiodes, etc. on the ML processor, the size of the single wafer can be further reduced.

[0118] For the sake of brevity of explanation, the method of the present disclosure is depicted and described as a series of actions. However, actions according to the present disclosure can occur in various orders and / or simultaneously, and occur together with other actions that are not proposed and described in the present disclosure. In addition, not all actions of explanation can be required to realize the method according to the disclosed subject matter. In addition, those skilled in the art will understand and recognize that the method can be represented as a series of mutual states via state diagrams or events alternatively.

[0119] As used herein, the terms "approximately," "about," and "substantially" may refer to within a normal tolerance range in the art, such as within 2 standard deviations of the mean, within ±20% of a target size in some embodiments, within ±10% of a target size in some embodiments, within ±5% of a target size in some embodiments, within ±2% of a target size in some embodiments, within ±1% of a target size in some embodiments, and within ±0.1% of a target size in some embodiments. The terms "approximately" and "about" may include a target size. Unless otherwise specified or apparent from the context, all numerical values ​​described herein are modified by the term "about."

[0120] As used herein, a range includes all values ​​within the range. For example, a range of 1 to 10 may include any number, combination of numbers, sub-ranges of numbers from 1, 2, 3, 4, 5, 6, 7, 8, 9 and 10, and fractions thereof.

[0121] The present disclosure has been described in detail in the above description. However, it is obvious that the present disclosure can be implemented without these specific details. In some examples, in order to highlight the content of the present invention, well-known structures and devices are shown in the form of block diagrams rather than specific details.

[0122] The terms "first", "second", "third", "fourth", etc. used in this document are marks used to distinguish different components and may not necessarily have the ordinal meaning of the numerical numbers used.

[0123] The word "example" or "exemplary" as used herein means serving as an example, instance or illustration. Any aspect or design described herein as "example" or "exemplary" is not necessarily to be understood as being more preferred or advantageous than other aspects or designs. On the contrary, the purpose of using the words "example" or "exemplary" is to present the concept in a concrete way. In this application, the term "or" means including "or" rather than excluding "or". That is, unless otherwise specified or as can be seen from the context, "X includes A or B" means any natural inclusive permutation and combination. That is, if X includes A; X includes B; or X includes both A and B, then in any of the above cases, "X includes A or B" is satisfied. In addition, "a" and "an" used in this application and the appended claims should generally be understood as "one or more", unless otherwise specified or as can be clearly indicated from the context as being directed to a singular form. "One embodiment" or "an embodiment" mentioned in this specification means that a specific feature, structure or characteristic associated with the embodiment is included in at least one embodiment. Therefore, the phrases "one embodiment" or "an embodiment" appearing in different places in this specification do not necessarily all refer to the same embodiment.

[0124] As used herein, when an element or layer is referred to as being “on” another element or layer, the element or layer may be directly on the other element or layer, or intervening elements or layers may be present. In contrast, when an element or layer is referred to as being “directly on” another element or layer, there are no intervening elements or layers present.

[0125] Although it is obvious to those skilled in the art that additional changes and modifications may be made to the present disclosure after understanding the above description, it should be understood that any specific embodiments shown and described by way of illustration should not be considered limiting. Therefore, the details of various embodiments are not intended to limit the scope of the claims, which themselves merely describe the disclosed technical features.

Claims

1. A semiconductor device, comprising: A sensing module configured to generate a plurality of analog sensing signals; Machine Learning (ML) processors, including: one or more crossbar arrays configured to process the analog sensor signals to generate analog pre-processed sensor data; an analog-to-digital converter (ADC) configured to convert the analog pre-processed sensor data into digital pre-processed sensor data; and A machine learning processing unit configured to process the digital pre-processed sensor data using one or more machine learning models, wherein the sensor module and the ML processor are fabricated on a single chip.

2. The semiconductor device according to claim 1, wherein the sensing module and the ML processor are fabricated on the same side of the wafer. 3 . The semiconductor device according to claim 2 , wherein the sensing module is manufactured on a first portion of the wafer, and the ML processor is manufactured on a second portion of the wafer surrounding the first portion of the wafer.

4. The semiconductor device according to claim 2, wherein the plurality of transistors of the ML processor and the plurality of sensors of the sensing module are fabricated on a substrate of the wafer, wherein the semiconductor device further comprises a plurality of interconnect layers fabricated on the plurality of transistors and the plurality of sensors.

5. The semiconductor device according to claim 4, wherein the plurality of sensors include a plurality of photodiodes, wherein a first metal via, a second metal via, and a third metal via in the first interconnect layer are respectively connected to a source region, a gate region, and a drain region of a first transistor in the plurality of transistors, and a fourth metal via and a fifth metal via in the first interconnect layer are connected to a first photodiode in the plurality of photodiodes.

6. The semiconductor device according to claim 5, wherein the RRAM device of the ML processor can be manufactured on a metal pad or a metal via on an interconnect layer among the plurality of interconnect layers, and the RRAM device is connected to the first transistor through the first interconnect layer. 7 . The semiconductor device according to claim 6 , wherein a microlens and a set of color filters in the sensing module are fabricated on the plurality of interconnection layers. 8 . The semiconductor device according to claim 2 , wherein the sensing module is fabricated on the ML processor. 9 . The semiconductor device according to claim 1 , wherein the sensing module and the ML processor are fabricated on different sides of the wafer. 10 . The semiconductor device of claim 1 , wherein the sensing module comprises an image sensor array, wherein the plurality of analog sensing signals comprises a plurality of analog image signals.

11. The semiconductor device according to claim 1, wherein the analog pre-processed sensing data corresponds to a plurality of features extracted from the analog sensing signal, and the machine learning processing unit performs machine learning using the extracted features. 12 . The semiconductor device of claim 1 , further comprising a packaging substrate, wherein the wafer is connected to the packaging substrate through an interconnect layer. 13 . The semiconductor device according to claim 1 , wherein the ML processor is powered using the analog sensing signal.

14. The semiconductor device according to claim 1, further comprising a transceiver configured to: sending a prediction output generated by the machine learning processing unit based on the one or more machine learning models to a computing device; and Instructions are received from the computing device to perform an operation based on the predicted output. 15 . The semiconductor device of claim 1 , wherein the analog pre-processed sensing data represents a convolution of an analog sensing signal and a convolution kernel. 16 . The semiconductor device of claim 15 , wherein conductance values ​​of a plurality of cross-point devices in the one or more crossbar arrays are programmed to represent a value of the convolution kernel. 17 . The semiconductor device of claim 1 , wherein the sensing module comprises a two-dimensional sensor array, wherein a plurality of cross-point devices in the one or more cross-point arrays are configured to receive as input analog sensing signals generated by the two-dimensional sensor array. 18 . The semiconductor device of claim 17 , wherein the one or more crossbar arrays include a plurality of crossbar arrays located in a plurality of different planes.

19. A semiconductor device comprising: A sensing module configured to generate a plurality of analog sensing signals; and a machine learning (ML) processor configured to generate a prediction output by processing the analog sensor signal using one or more machine learning models, wherein the machine learning processor comprises: a plurality of crossbar arrays configured to generate a plurality of simulated outputs representing predicted outputs; An analog-to-digital conversion unit is configured to convert the plurality of analog outputs representing the predicted outputs into a plurality of digital signals representing the predicted outputs, wherein the sensing module and the ML processor are manufactured on a chip.

20. The semiconductor device of claim 19, further comprising a packaging substrate, wherein the die is connected to the packaging substrate through an interconnect layer.

Images