Semiconductor package for SRAM and npu

Abstract

According to one example of the present disclosure, an NPU semiconductor package is provided. The NPU semiconductor package may include: a semiconductor package substrate; an NPU chip mounted on the semiconductor package substrate, rotated at a first angle relative to one side of the semiconductor package substrate with respect to a plane; and one or more Static Random Access Memory (SRAM) chips mounted on the semiconductor package substrate adjacent to the NPU chip, rotated at a second angle relative to one side of the semiconductor package substrate with respect to a plane. The first angle and the second angle may be determined such that the size of the semiconductor package substrate is smaller than the size of a specific form factor.

Description

Technology Field

[0001] The present disclosure relates to a semiconductor package. Background Technology

[0002] Driven by the rapid advancement of the electronics industry and user demands, electronic devices are becoming increasingly smaller and lighter. Consequently, the semiconductor packages used in these devices are also becoming smaller and lighter; furthermore, high reliability is required for these packages, along with high performance and large capacity.

[0003] Concerns are spreading regarding the sustainability of rapidly advancing AI. Semiconductors centered on GPUs and HBM memory are placing limitations on the sustained development of AI due to factors such as the surge in power consumption in AI data centers and the resulting increase in costs. For example, the power demand of global data centers is exceeding that of a single developed nation. As the demand for AI semiconductors increases, it is predicted that the supply capacity of critical infrastructure, such as power plants and power supply chains, will be put at risk.

[0004] As AI has recently spread across all industries, on-device AI, which provides AI services directly on the device, is gaining prominence and creating a new market for AI semiconductors.

[0005] On-device AI services are being actively developed in fields such as automobiles, autonomous robots, drones, extended reality (XR), energy, intelligent home appliances, wearable healthcare, digital manufacturing, distribution / logistics, CCTV, and defense.

[0006] Various inference services are being developed for on-device AI services, and various neural network models are being developed for academia, content, traffic control, law, education, CCTV, and telecommunication base stations.

[0007] Providing AI services on-device allows them to be optimized for personalized services. However, implementing AI services on-device requires overcoming several challenges arising from environments different from data centers. First, it must be possible to provide ultra-lightweight, high-performance AI models that offer performance comparable to that of massive AI models. The problem to be solved

[0008] In order to provide AI services on-device, there may be physical size limitations on AI semiconductors. On-devices can be mounted on wearable devices, smart devices, bipedal robots, quadrupedal robots, drones, etc., and the physical size of the AI ​​semiconductor can be very important.

[0009] Providing on-device AI services may be subject to power consumption limitations for AI semiconductors. High on-device power consumption can lead to an excessively short operating time for battery-powered devices. Therefore, reducing the power consumption of on-device AI semiconductors can be critical.

[0010] To provide AI services on-device, it is necessary to overcome various technical challenges.

[0011] First, bandwidth between the processor and memory must be guaranteed. If bandwidth between the on-device processor and memory is not guaranteed, it can have various impacts on AI computation performance. However, the memory available for on-device use may be limited. Therefore, server-grade HBM (High Bandwidth Memory) faces difficulties in adoption on-devices due to its large size and high power consumption. Additionally, when the on-device is battery-powered, DRAM (Dynamic Random Access Memory) has the disadvantage of reducing battery life.

[0012] Second, the physical distance between the processor and memory must be minimized. An increase in the physical distance between the on-device processor and memory can have various effects on power consumption.

[0013] For example, the problem of increased power consumption arises. As physical distances increase, the resistance and capacitance of the interconnections between the processor and memory rise, leading to increased power consumption. Since this increased resistance requires more energy to transmit signals over longer distances, higher power loss occurs in the form of heat.

[0014] For example, signal integrity issues may arise. As distance increases, signal integrity problems such as signal degradation and increased sensitivity to interference may occur. To maintain signal integrity, the system may need to raise voltage levels or use more sophisticated signaling techniques, both of which can lead to higher power consumption.

[0015] For example, the need for signal amplification arises. To maintain signal integrity over long distances, signal amplification mechanisms such as repeaters or amplifiers may be required. Since these additional components consume power, they further contribute to the overall system power consumption.

[0016] For example, the issue of increased latency arises. If the distance between the processor and memory is long, latency can increase because it takes longer for signals to travel between the two components. Increased latency can lead to inefficient system operation and potentially increase power consumption as the processor has to wait longer for data.

[0017] For example, a problem arises regarding increased dynamic power. The dynamic power consumption associated with the switching activity of transistors can increase as the distance increases. This is because parasitic capacitance increases with longer interconnections, requiring more energy to charge and discharge during each switching cycle.

[0018] Accordingly, the inventor of the present disclosure has conducted various studies on semiconductor packaging that can be applied to on-devices.

[0019] Accordingly, the present disclosure aims to provide a semiconductor package for an NPU and NPU-dedicated memory. means of solving the problem

[0020] To achieve the aforementioned objectives, according to one example of the present disclosure, an NPU semiconductor package is provided. The NPU semiconductor package may include: a semiconductor package substrate; an NPU chip mounted on the semiconductor package substrate, rotated at a first angle relative to one side of the semiconductor package substrate with respect to a plane; and one or more SRAM chips mounted on the semiconductor package substrate adjacent to the NPU chip, rotated at a second angle relative to one side of the semiconductor package substrate with respect to a plane. The first angle and the second angle may be determined such that the size of the semiconductor package substrate is smaller than the size of a specific form factor.

[0021] According to one example of the present disclosure, an NPU semiconductor package is also provided. The NPU semiconductor package may include: a substrate; an NPU mounted on the substrate, rotated at a first angle relative to one side of the substrate with respect to a plane; and one or more SRAMs mounted on the substrate adjacent to the NPU, rotated at a second angle relative to one side of the substrate with respect to a plane. The first angle and the second angle may be adjusted so that the size of the substrate is smaller than the size of a specific form factor.

[0022] According to one example of the present disclosure, an AI processor package is provided. The AI ​​processor package may include an AI processor specialized for AI computation; and one or more SRAMs electrically connected to the AI ​​processor. The AI ​​processor and the one or more SRAMs may be arranged in a rotated multi-chip substrate layout to form a rotated multi-chip substrate package.

[0023] To achieve the aforementioned objective, according to one example of the present disclosure, a Neural Processing Unit (NPU) semiconductor package is provided. The NPU semiconductor package may include a semiconductor package substrate having solder bumps formed on its lower surface; an NPU mounted on the semiconductor package substrate in a diagonal direction opposite to the semiconductor package substrate; and a plurality of SRAMs mounted on the semiconductor package substrate to surround the NPU in the diagonal direction. Effects of the invention

[0024] According to one example of the present disclosure, an NPU semiconductor package equipped with dedicated memory can be miniaturized.

[0025] According to one example of the present disclosure, a high memory bandwidth can be reliably guaranteed to the NPU semiconductor through a dedicated memory provided inside a package.

[0026] According to one example of the present disclosure, parameters of a large-capacity neural network model can be reliably supplied to an NPU semiconductor through dedicated memory in an on-device environment.

[0027] According to one example of the present disclosure, an NPU semiconductor package equipped with dedicated memory can be miniaturized to facilitate the commercialization of an edge device dedicated to AI inference. Brief explanation of the drawing

[0028] Figure 1 is a schematic conceptual diagram illustrating an exemplary neural network model. Figure 2a is a diagram illustrating the basic structure of a convolutional neural network (CNN). Figure 2b is a comprehensive diagram that makes it easy to understand the operation of a convolutional neural network. FIG. 3 is a schematic conceptual diagram illustrating a neural processing unit according to one example of the present disclosure. FIG. 4a is a schematic conceptual diagram illustrating one of a processing element among a plurality of processing elements that can be applied to one example of the present disclosure. FIG. 4b is a schematic conceptual diagram illustrating a special function unit (SFU) that can be applied to one example of the present disclosure. FIG. 5 is an illustrative diagram showing a variation of the neural processing unit (100) illustrated in FIG. 3. Figure 6 is a block diagram showing a system according to one example. FIGS. 7A and 7B are cross-sectional views schematically showing the pre- and post-packaging of a fan-in semiconductor package. Figure 8 is a cross-sectional view schematically illustrating the packaging process of a fan-in semiconductor package. FIG. 9A is a schematic cross-sectional view showing a case where a fan-in semiconductor package is mounted on an interposer substrate and finally mounted on a main board of an electronic device, and FIG. 9B is a schematic cross-sectional view showing a case where a fan-in semiconductor package is embedded within an interposer substrate and finally mounted on a main board of an electronic device. Figure 10 is a cross-sectional view showing a schematic appearance of a fan-out semiconductor package. FIG. 11 is a schematic cross-sectional view showing a case where a fan-out semiconductor package is mounted on a main board of an electronic device. FIG. 12 is a cross-sectional view showing a package-on-package (PoP) according to the first example. FIGS. 13a to 13d are cross-sectional views illustrating a method for manufacturing a package-on-package (PoP) (3000) illustrated in FIG. 12. FIG. 14 is a cross-sectional view showing a package-on-package (PoP) according to the second example. FIGS. 15a to 15e describe a method for manufacturing a package-on-package (PoP) (3000b) illustrated in FIG. 14. FIG. 16 is a cross-sectional view showing a package-on-package (PoP) according to the third example. FIG. 17 is a cross-sectional view showing a package-on-package (PoP) according to the fourth example. FIG. 18 is a cross-sectional view showing a package according to the fifth example. FIG. 19 is a cross-sectional view showing a package-on-package (PoP) according to the 6th example. FIG. 20 is a schematic perspective view illustrating a semiconductor package (4000) according to a 2.5D package method. FIG. 21 is a schematic cross-sectional view of the semiconductor package (4000) shown in FIG. 20. FIG. 22 is a schematic top view showing the top surface of the interposer (4020) illustrated in FIG. 20. FIG. 23 is a schematic bottom view of the first and second base dies (4050A, 4050B) of the first and second memory stacks (4040A, 4040B) shown in FIG. 20. FIG. 24 is a schematic top view of the first and second base dies (4050A, 4050B) of the first and second memory stacks (4040A, 4040B) shown in FIG. 20. Figure 25 shows an example of a form factor with an M.2 slot. FIG. 26 is an exemplary diagram showing a package (5000) according to the first method of the present disclosure. FIG. 27 is an exemplary diagram showing a package (5000) according to the second method of the present disclosure. FIG. 28 is an example diagram showing a cross-sectional view along the line A-A' of the package (5000) illustrated in FIG. 26 or FIG. 27. Specific details for implementing the invention

[0029] Specific structural or step-by-step descriptions of the examples of the present disclosure are merely for the purpose of illustrating examples according to the concept of the present disclosure. Accordingly, examples according to the concept of the present disclosure may be implemented in various forms. Examples according to the concept of the present disclosure may be implemented in various forms. The present disclosure shall not be interpreted as being limited to the examples of the present disclosure.

[0030] Various modifications may be made to the embodiments according to the concept of the present disclosure and may take various forms. Accordingly, specific embodiments are illustrated in the drawings and the present disclosure or application is to be described in detail. However, this is not intended to limit the embodiments according to the concept of the present disclosure to specific forms of disclosure. It should be understood that the embodiments according to the concept of the present disclosure include all modifications, equivalents, and substitutions that fall within the spirit and scope of the present disclosure.

[0031] Terms such as "first" and / or "second" may be used to describe various components, but said components should not be limited by said terms. Such terms may be used solely for the purpose of distinguishing one component from another. Without departing from the scope of rights according to the concept of the present disclosure, said terms may refer to the first component as the second component, and similarly, the second component as the first component.

[0032] When it is stated that one component is "connected" or "connected" to another component, it may be directly connected to or connected to that other component. However, it should be understood that another component may exist between the multiple components. Conversely, when it is stated that one component is "directly connected" or "directly connected" to another component, it should be understood that no other component exists between them. Other expressions describing the relationship between components, such as "between" and "exactly between," or "adjacent to" and "directly adjacent to," should be interpreted in the same way.

[0033] The terms used in this disclosure are for illustrative purposes only and are not intended to limit this disclosure. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this disclosure, terms such as “comprising” or “having” are intended to indicate the existence of the described features, numbers, steps, actions, components, parts, or combinations thereof. Accordingly, it should be understood that the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof is not excluded in advance.

[0034] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as generally understood by those skilled in the art to which this disclosure pertains. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology. Terms such as those defined in commonly used dictionaries are not to be interpreted in an ideal or overly formal sense unless explicitly defined in this disclosure.

[0035] In describing the examples, technical details that are well known in the technical field to which this disclosure belongs and are not directly related to this disclosure are omitted. This is intended to convey the essence of this disclosure more clearly without obscuring it by omitting unnecessary explanations.

[0036] Definition of Terms

[0037] Hereinafter, in order to facilitate understanding of the disclosures presented in this disclosure, the terms used in this disclosure will be briefly summarized.

[0038] NPU: Abbreviation for Neural Processing Unit, it refers to a processor specialized for the computation of neural network models, separate from the CPU (Central Processing Unit).

[0039] NN: Abbreviation for neural network, which can refer to a network in which nodes are connected in a layered structure by mimicking how neurons in the human brain are connected through synapses in order to mimic human intelligence.

[0040] DNN: Abbreviation for Deep Neural Network, which can mean increasing the number of hidden layers of an artificial neural network to achieve higher artificial intelligence.

[0041] CNN: Abbreviation for Convolutional Neural Network, it is a neural network that performs functions similar to how the visual cortex of the human brain processes images. Convolutional neural networks are known to be suitable for image processing and are known to be effective at extracting features from input data and identifying patterns among those features.

[0042] Dedicated memory: Dedicated memory provided for data processed by the NPU, which can be configured to store the parameters of a neural network model and provide the NPU with the data necessary for processing the neural network model in real time. The dedicated memory is packaged together with the NPU and can improve communication quality (e.g., signal-to-noise ratio) by minimizing the length of the data transmission path.

[0043] The present disclosure is described in detail below by explaining preferred examples of the present disclosure with reference to the attached drawings. Hereinafter, examples of the present disclosure are described in detail with reference to the attached drawings.

[0044] Artificial Intelligence

[0045] Humans possess intelligence capable of recognition, classification, inference, prediction, and control / decision-making. Artificial intelligence (AI) refers to the artificial imitation of human intelligence.

[0046] The human brain is composed of countless nerve cells called neurons. Each neuron is connected to hundreds to thousands of other neurons through connection points called synapses. A neural network model is a model that replicates the operating principles of biological neurons and the relationships between them in order to mimic human intelligence. In other words, an artificial neural network is a system that connects nodes, which mimic neurons, in a layered structure.

[0047] These neural network models are classified into 'single-layer neural networks' and 'multi-layer neural networks' depending on the number of layers. A general multi-layer neural network consists of an input layer, a hidden layer, and an output layer. (1) The input layer is a layer that receives external data, and the number of neurons in the input layer is equal to the number of input variables. (2) The hidden layer is located between the input layer and the output layer, receives signals from the input layer, extracts features, and transmits them to the output layer. (3) The output layer receives signals from the hidden layer and outputs them externally. Input signals between neurons are multiplied by their respective connection strengths, which have values ​​between 0 and 1, and then summed. If this sum is greater than the neuron's threshold, the neuron is activated and implemented as an output value through an activation function.

[0048] Meanwhile, to achieve higher artificial intelligence, an artificial neural network with an increased number of hidden layers is called a Deep Neural Network (DNN).

[0049] DNNs are being developed in various structures. For example, the Convolutional Neural Network (CNN), an example of a DNN, is known to be effective for extracting features from input values ​​(videos or images) and identifying patterns in the extracted output values. CNNs can be configured such that convolution operations, activation function operations, and pooling operations are processed in a specific order.

[0050] For example, in each layer of a DNN, parameters (i.e., input values, output values, weights, or kernels, etc.) can be matrices composed of multiple channels. The parameters can be processed by a Neural Processing Unit (NPU) through convolution or matrix multiplication. An output value is generated from the computations performed in each layer.

[0051] For example, the Transformer model is a DNN based on attention technology. Transformers utilize numerous matrix multiplication operations. Transformers can obtain an output value called attention (Q,K,V) using input values ​​and parameters such as query (Q), key (K), and value (V). Transformers can process various inference operations based on the output value (i.e., attention (Q,K,V)). Transformers tend to demonstrate superior inference performance compared to CNNs.

[0053] Figure 1 is a schematic conceptual diagram illustrating an exemplary neural network model.

[0054] The following describes the operation of an exemplary neural network model (110a) that can be operated in a neural processing unit (100).

[0055] The exemplary neural network model (110a) of FIG. 1 may be an artificial neural network trained to perform various inference functions such as object recognition and speech recognition.

[0056] The neural network model (110a) may be a deep neural network (DNN).

[0057] However, the neural network model (110a) according to the examples of the present disclosure is not limited to a deep neural network.

[0058] For example, the neural network model (110a) may be implemented with models such as Chat-GPT, LLaMA, Stable Diffusion, LLM, Generative Adversarial Networks (GAN), Florence, DaViT, MobileViT, Swin-Transformer, Transformer, YOLO, CNN, PIDNet, BiseNet, RCNN, VGG, VGG16, DenseNet, SegNet, DeconvNet, DeepLAB V3+, U-net, SqueezeNet, AlexNet, ResNet18, MobileNet-v2, GoogLeNet, Resnet-v2, Resnet50, Resnet101, Inception-v3, etc. However, the present disclosure is not limited to the models described above. Additionally, the neural network model (110a) may be a multimodal or an ensemble model based on at least two different models.

[0059] The following describes the inference process performed by an exemplary neural network model (110a).

[0060] The neural network model (110a) is an exemplary deep neural network model comprising an input layer (110a-1), a first network (110a-2), a first hidden layer (110a-3), a second network (110a-4), a second hidden layer (110a-5), a third network (110a-6), and an output layer (110a-7). However, the present disclosure is not limited to the neural network model illustrated in FIG. 1. The first hidden layer (110a-3) and the second hidden layer (110a-5) may also be referred to as a plurality of hidden layers.

[0061] The input layer (110a-1) may, for example, include x1 and x2 input nodes. That is, the input layer (110a-1) may include information about two input values.

[0062] The first network (110a-2) may, for example, include information on six weight values ​​for connecting each node of the input layer (110a-1) to each node of the first hidden layer (110a-3). Each weight value is multiplied by an input node value, and the accumulated value of the multiplied values ​​is stored in the first hidden layer (110a-3). The weight values ​​and input node values ​​may be referred to as parameters of the neural network model.

[0063] The first hidden layer (110a-3) may include, for example, nodes a1, a2, and a3. That is, the first hidden layer (110a-3) may include information about three node values.

[0064] The first processing element (PE1) of Fig. 1 can process the operation of node a1.

[0065] The second processing element (PE2) of Fig. 1 can process the operation of node a2.

[0066] The third processing element (PE3) of FIG. 1 can process the operation of node a3. The second network (110a-4) may, for example, include information on nine weight values ​​for connecting each node of the first hidden layer (110a-3) to each node of the second hidden layer (110a-5). The weight values ​​of the second network (110a-4) are each multiplied with the node values ​​input from the first hidden layer (110a-3), and the accumulated value of the multiplied values ​​is stored in the second hidden layer (110a-5).

[0067] The second hidden layer (110a-5) may include, for example, nodes b1, b2, and b3. That is, the second hidden layer (110a-5) may include information about three node values.

[0068] The fourth processing element (PE4) of Fig. 1 can process the operation of the b1 node.

[0069] The fifth processing element (PE5) of Fig. 1 can process the operation of the b2 node.

[0070] The 6th processing element (PE6) of Fig. 1 can process the operation of the b3 node.

[0071] The third network (110a-6) may, for example, include information on six weight values ​​connecting each node of the second hidden layer (110a-5) and each node of the output layer (110a-7). The weight values ​​of the third network (110a-6) are each multiplied by the node values ​​input from the second hidden layer (110a-5), and the accumulated value of the multiplied values ​​is stored in the output layer (110a-7).

[0072] The output layer (110a-7) may include, for example, nodes y1 and y2. That is, the output layer (110a-7) may include information about the two node values.

[0073] The seventh processing element (PE7) of Fig. 1 can process the operation of the y1 node.

[0074] The eighth processing element (PE8) of Fig. 1 can process the operation of the y2 node.

[0075] Each node can correspond to a feature value, and the feature value can correspond to a feature map.

[0077] Figure 2a is a diagram illustrating the basic structure of a convolutional neural network (CNN).

[0078] Referring to FIG. 2a, the input image can be represented as a two-dimensional matrix consisting of rows of a specific size and columns of a specific size. The input image may have multiple channels, where the channels represent the number of color components of the input data image.

[0079] The convolution process refers to performing convolution operations with kernels while traversing the input image at specified intervals.

[0080] Convolutional neural networks can have a structure that passes the output value of the current layer (convolution or matrix multiplication) as the input value to the next layer.

[0081] For example, convolution is defined by two main parameters (input feature map and kernel). The parameters of a neural network model may include input feature maps, output feature maps, activation maps, weights, kernels, and attention (Q, K, V), etc.

[0082] Convolution slides a kernel window over an input feature map. The step size over which the kernel slides across the input feature map is called the stride.

[0083] Pooling can be applied after convolution. Additionally, fully-connected (FC) layers can be placed at the ends of the convolutional neural network.

[0084] However, the neural network model of the present disclosure is not limited to CNN.

[0085] Figure 2b is a comprehensive diagram that makes it easy to understand the operation of a convolutional neural network.

[0086] Referring to FIG. 2b, the input image is shown as an example of a two-dimensional matrix having a size of 6 x 6. Also, FIG. 2b shows that three nodes, namely Channel 1, Channel 2, and Channel 3, are used as an example.

[0087] First, I will explain convolutional operations.

[0088] The input image (exemplarily shown as being 6 x 6 in FIG. 2b) is convolutional with kernel 1 for channel 1 (exemplarily shown as being 3 x 3 in FIG. 2b) at the first node, and as a result, feature map 1 (exemplarily shown as being 4 x 4 in FIG. 2b) is output. Additionally, the input image (exemplarily shown as being 6 x 6 in FIG. 2b) is convolutional with kernel 2 for channel 2 (exemplarily shown as being 3 x 3 in FIG. 2b) at the second node, and as a result, feature map 2 (exemplarily shown as being 4 x 4 in FIG. 2b) is output. In addition, the input image is convolutioned with kernel 3 for channel 3 (shown as 3 x 3 in FIG. 2b, for example) at the third node, and as a result, feature map 3 (shown as 4 x 4 in FIG. 2b, for example) is output.

[0089] To process each convolution, the processing elements (PE1 to PE12) of the neural processing unit (100) are configured to perform MAC operations.

[0090] Next, we will explain the operation of the activation function.

[0091] An activation function can be applied to feature map 1, feature map 2, and feature map 3 (each shown as having an exemplary size of 4 x 4 in FIG. 2b) output from a convolution operation. The output after the activation function is applied can have an exemplary size of 4 x 4.

[0092] Next, we will explain the pooling operation.

[0093] Feature Map 1, Feature Map 2, and Feature Map 3 output from the above activation function (each shown as having an exemplary size of 4 x 4 in Fig. 2b) are input to three nodes. Pooling can be performed by receiving the feature maps output from the activation function as input. Pooling can reduce the size or emphasize specific values ​​within a matrix. Pooling methods include maximum value pooling, average pooling, and minimum value pooling. Maximum value pooling is used to collect the maximum value within a specific region of the matrix, and average pooling can be used to calculate the average within a specific region.

[0094] In the example of Figure 2b, a 4 x 4 feature map is shown to be reduced to a 2 x 2 size by polling.

[0095] Specifically, the first node takes feature map 1 for channel 1 as input, performs polling, and outputs, for example, a 2 x 2 matrix. The second node takes feature map 2 for channel 2 as input, performs polling, and outputs, for example, a 2 x 2 matrix. The third node takes feature map 3 for channel 3 as input, performs polling, and outputs, for example, a 2 x 2 matrix.

[0096] The aforementioned convolution, activation function, and polling are repeated, and finally, the result can be output to a fully connected layer. The output can then be input back into an artificial neural network for image recognition. However, the present disclosure is not limited to the size of the feature map or kernel.

[0097] The CNN described so far is the most widely used method in the field of computer vision among various deep neural network (DNN) methods. In particular, CNNs have demonstrated remarkable performance in various research areas performing diverse tasks such as image classification and object detection.

[0098] However, the neural network model of the present disclosure is not limited to CNN, and the NPU can process various neural network models.

[0100] <ANN의 연산을 위해 필요한 하드웨어 자원>

[0101] FIG. 3 is a schematic conceptual diagram illustrating a neural processing unit according to one example of the present disclosure.

[0102] The neural processing unit (NPU) (100) illustrated in FIG. 3 is a processor specialized to perform operations for artificial neural networks.

[0103] An artificial neural network refers to a network composed of artificial neurons that, upon receiving multiple inputs or stimuli, multiply each by a weight, add the results, and then transform the added deviation through an activation function before transmitting the result. An artificial neural network trained in this way can be used to output inference results from input data.

[0104] The neural processing unit (100) may be a semiconductor implemented as an electric / electronic circuit. The term electric / electronic circuit may mean one that includes numerous electronic components (e.g., transistors, capacitors).

[0105] In the case of a Transformer and / or CNN-based neural network model, the neural processing unit (100) can select and process matrix multiplication operations, convolution operations, etc. according to the architecture of the artificial neural network.

[0106] For example, in each layer of a Convolutional Neural Network (CNN), the input feature map corresponding to the input data and the kernel corresponding to the weight can be a tensor or matrix composed of multiple channels. A convolution operation is performed between the input feature map and the kernel, and an output feature map is generated for each channel through the convolution operation and pooling. An activation function is applied to the output feature map to generate the activation map for the corresponding channel. Subsequently, pooling can be applied to the activation map. Here, the activation map can be broadly referred to as the output feature map. For the convenience of explanation below, the activation map will be referred to as the output feature map.

[0107] However, the examples of the present disclosure are not limited thereto, and the output feature map means that matrix multiplication or convolution operations, etc. have been applied.

[0108] To elaborate, the output feature map according to the examples of the present disclosure should be interpreted in a comprehensive sense. For example, the output feature map may be the result of a matrix multiplication operation or a convolution operation. Accordingly, the plurality of processing elements (110) may be modified to further include a processing circuit for additional algorithms. That is, it is possible to configure some circuit units of the SFU (150) described later to be included in the plurality of processing elements (110).

[0109] The neural processing unit (100) may be configured to include a plurality of processing elements (110) for processing convolution and matrix multiplication required for the artificial neural network operation described above.

[0110] The neural processing unit (100) may be configured to include respective processing circuits optimized for matrix multiplication, convolution, activation function, pooling, stride, batch normalization, skip connection, joining, quantization, inverse quantization, softmax function, clipping, and padding operations required for the artificial neural network operations described above.

[0111] For example, the neural processing unit (100) may be configured to include an SFU (150) for processing at least one of the activation function operation, pooling operation, stride operation, batch normalization operation, skip connection operation, joining operation, quantization operation, clipping operation, and padding operation among the algorithms described above.

[0112] Specifically, the neural processing unit (100) may include a plurality of processing elements (PE) (110), an SFU (150), an NPU internal memory (120), an NPU controller (130), and an NPU interface (140). Each of the plurality of processing elements (110), the SFU (150), the NPU internal memory (120), the NPU controller (130), and the NPU interface (140) may be a semiconductor circuit with numerous transistors connected thereto. Therefore, some of these may be difficult to identify and distinguish visually and may only be identifiable by operation.

[0113] For example, any circuit may operate as a plurality of processing elements (110) or as an NPU controller (130). The NPU controller (130) may be configured to perform the function of a control unit configured to control the artificial neural network inference operation of the neural processing unit (100).

[0114] The neural processing unit (100) may include an NPU internal memory (120) configured to store parameters of a neural network model that can be inferred from a plurality of processing elements (110) and an SFU (150), and an NPU controller (130) configured to control the operation schedule of the plurality of processing elements (110), the SFU (150), and the NPU internal memory (120).

[0115] The neural processing unit (100) may be configured to process feature maps corresponding to encoding and decoding methods using SVC (scalable video coding) or SFC (scalable feature-map coding). These methods are techniques that vary the amount of data transmitted variably according to the effective bandwidth and signal-to-noise ratio (SNR) of a communication channel or communication bus. That is, the neural processing unit (100) may also be configured to include an encoder and a decoder.

[0116] Multiple processing elements (110) can perform part of the operation for the artificial neural network.

[0117] SFU(150) can perform other parts of the operation for the artificial neural network.

[0118] The neural processing unit (100) can be configured to hardware accelerate the computation of a neural network model using a plurality of processing elements (110) and an SFU (150).

[0119] The NPU interface (140) can communicate with various components, such as memory, connected to the neural processing unit (100) via the system bus. Here, the memory may be dedicated memory for the NPU. Here, the dedicated memory may be a memory circuit included in the same package as the NPU.

[0120] The NPU controller (130) may be configured to control the operation of a plurality of processing elements (110) for inference operations of the neural processing unit (100), the operation of the SFU (150), and the read and write order of the NPU internal memory (120). For example, the NPU controller (130) may be configured to schedule neural network operations or to control each circuit of the neural processing unit (100) in a scheduled order.

[0121] The NPU controller (130) may be configured to control a plurality of processing elements (110), SFU (150), and NPU internal memory (120) based on information regarding data locality or structure of the neural network model.

[0122] The NPU controller (130) may analyze the structure of a neural network model to be operated on a plurality of processing elements (110) and SFU (150), or receive information that has already been analyzed. The analyzed information may be information generated by a compiler. For example, the artificial neural network data that the neural network model may include may include at least some of the node data of each layer (i.e., feature map), the arrangement data of the layers, locality information or information about the structure, and the weight data of each network connecting the nodes of each layer (i.e., weight kernel). The artificial neural network data may be stored in memory provided within the NPU controller (130) or in the NPU internal memory (120). However, it is not limited thereto, and the artificial neural network data may be stored in a separate cache memory or register file provided in the NPU or the SoC (System on Chip) including the NPU.

[0123] The NPU controller (130) can schedule the order of operations of the neural network model to be performed by the neural processing unit (100) based on information about the data locality or structure of the neural network model.

[0124] The NPU controller (130) may receive scheduling information for the order of operations of the neural network model to be performed by the neural processing unit (100) based on information regarding data locality information or structure of the compiled neural network model. For example, the scheduling information may be information generated by the compiler. The scheduling information generated by the compiler may be referred to as machine code or binary code, etc.

[0125] That is, the scheduling information used in the NPU controller (130) may be information generated by a compiler based on the data locality information or structure of the neural network model.

[0126] To elaborate, the compiler can efficiently schedule the NPU depending on how well it understands and reconstructs the artificial neural network data locality, which is an inherent characteristic of neural network models.

[0127] To elaborate, the compiler can efficiently schedule the NPU depending on how well it understands the hardware structure and performance of the neural processing unit (100).

[0128] To elaborate, when a neural network model is compiled by a compiler to be executed in a neural processing unit (100), the artificial neural network data locality can be reconfigured. The artificial neural network data locality can be reconfigured according to the algorithms applied to the neural network model and the operating characteristics of the processor.

[0129] To elaborate, the artificial neural network data locality can be reconfigured according to the way the neural processing unit (100) processes the neural network model, for example, feature map tiling, or the stationary method of the processing element.

[0130] To elaborate, the artificial neural network data locality can be reconfigured according to the number of processing elements of the neural processing unit (100) and the capacity of the internal memory.

[0131] To elaborate, the artificial neural network data locality can be reconfigured according to the bandwidth of the memory communicating with the neural processing unit (100).

[0132] This is because, even if the same neural network model is processed by each of the factors described above, the neural processing unit (100) can determine the order of the necessary data differently at every moment in clock units.

[0133] The compiler can determine data locality based on the order of operations of the artificial neural network layers, identity convolutions, and / or matrix multiplications for the order of data required for the operation of the neural network model, and generate compiled machine code.

[0134] The compiler can be separate software running outside the NPU or an embedded compiler running inside the NPU.

[0135] The NPU controller (130) can be configured to utilize scheduling information included in the machine code.

[0136] The NPU controller (130) can obtain a memory address value in which feature maps and weight data of layers of a neural network model are stored based on scheduling information. Here, the address value may include an address value of the NPU internal memory and / or an address value of a dedicated memory.

[0137] For example, the NPU controller (130) can obtain a memory address value in which feature maps and weight data of a layer of a neural network model are stored in memory (e.g., NPU internal memory or dedicated memory). Thus, the NPU controller (130) can retrieve activation parameters and weight parameters of a layer of the neural network model to be executed from dedicated memory and store them in the NPU internal memory (120).

[0138] The activation parameter of each layer can have a corresponding memory address value.

[0139] Each weight parameter can have a corresponding memory address value.

[0140] The NPU controller (130) may receive scheduling information regarding the operation order of a plurality of processing elements (110) based on information regarding data locality information or structure of the neural network model, for example, batch data of the layers of the artificial neural network of the neural network model, locality information or structure information. The scheduling order may be determined at the compilation stage or in real time.

[0141] The NPU controller (130) operates based on information scheduled based on data locality information or information about the structure of the neural network model, so it can operate differently from the general CPU scheduling concept. General CPU scheduling operates to achieve the best efficiency by considering fairness, efficiency, stability, response time, etc. That is, it schedules to perform the most processing within the same time by considering priority, computation time, etc.

[0142] Conventional CPUs used algorithms that scheduled tasks by considering data such as the priority of each processing step and computation time.

[0143] Alternatively, the NPU controller (130) can control the neural processing unit (100) in the processing order of the neural processing unit (100) determined based on information about the data locality or structure of the neural network model.

[0144] Furthermore, the NPU controller (130) can drive the NPU (100) in a processing order determined based on information about the data locality information or structure of the neural network model and / or information about the data locality information or structure of the neural processing unit (100) to be used.

[0145] However, the present disclosure is not limited to information on data locality or structure of the neural processing unit (100).

[0146] The NPU controller (130) may be configured to store information about the data locality or structure of the artificial neural network.

[0147] That is, the NPU controller (130) can determine the processing order even if it only utilizes information about the data locality or structure of the artificial neural network of the neural network model.

[0148] Furthermore, the NPU controller (130) can determine the processing order of the neural processing unit (100) by considering the data locality information or structure information of the neural network model and the data locality information or structure information of the neural processing unit (100). Additionally, the processing of the neural processing unit (100) can be optimized according to the determined processing order.

[0149] That is, the NPU controller (130) may be configured to operate based on machine code compiled from a compiler, but in other examples, the NPU controller (130) may also be configured to have an embedded compiler. According to the above configuration, the neural processing unit (100) may be configured to receive a file in the format of a framework of various AI software and generate machine code. For example, frameworks of AI software include TensorFlow, PyTorch, Keras, XGBoost, mxnet, DARKNET, ONNX, etc.

[0150] A plurality of processing elements (110) refers to a configuration in which a plurality of processing elements (PE1 to PE12) configured to compute feature maps and weight data of an artificial neural network are arranged. Each processing element may include a MAC (multiply and accumulate) operator and / or an ALU (Arithmetic Logic Unit) operator. However, examples according to the present disclosure are not limited thereto.

[0151] Each processing element can be configured to optionally include additional special function units to handle additional special functions.

[0152] For example, the processing element (PE) may be modified to further include a batch-normalization unit, an activation function unit, an interpolation unit, etc.

[0153] The SFU (150) may include a circuit section configured to select and process activation function operations, pooling operations, stride operations, batch-normalization operations, skip-connection operations, concatenation operations, quantization operations, clipping operations, padding operations, etc., according to the architecture of the artificial neural network. That is, the SFU (150) may include a plurality of special function operation processing circuit units.

[0154] Although multiple processing elements are illustrated as examples in FIG. 3, it is also possible to configure a processing element by replacing the MAC within a single processing element with a plurality of multipliers and an adder tree implemented in parallel. In this case, the plurality of processing elements (110) may be referred to as at least one processing element including a plurality of arithmetic units.

[0155] A plurality of processing elements (110) is configured to include a plurality of processing elements (PE1 to PE12). The plurality of processing elements (PE1 to PE12) shown in FIG. 3 are merely examples for convenience of explanation, and the number of the plurality of processing elements (PE1 to PE12) is not limited. The size or number of the plurality of processing elements (110) may be determined by the number of the plurality of processing elements (PE1 to PE12). The size of the plurality of processing elements (110) may be implemented in the form of an N x M matrix. Here, N and M are integers greater than 0. The plurality of processing elements (110) may include N x M processing elements. That is, there may be one or more processing elements.

[0156] The size of the multiple processing elements (110) can be designed by taking into account the characteristics of the neural network model in which the neural processing unit (100) operates.

[0157] Multiple processing elements (110) are configured to perform functions such as addition, multiplication, and accumulation required for artificial neural network operations. Alternatively, multiple processing elements (110) may be configured to perform MAC (multiplication and accumulation) operations.

[0158] The first processing element (PE1) among the multiple processing elements (110) is described below as an example.

[0159] The neural processing unit (100) according to the examples of the present disclosure may be referred to as an AI semiconductor chip, an AI accelerator, a neural processor chip, etc.

[0160] FIG. 4a is a schematic conceptual diagram illustrating one processing element that can be applied to one example of the present disclosure.

[0161] Referring to FIG. 4a, the processing element (PE) may be configured to include a multiplier (111), an adder (112), an accumulator (113), and a bit quantization unit (114). However, the examples according to the present disclosure are not limited thereto, and a plurality of processing elements may be modified in consideration of the computational characteristics of the neural network model.

[0162] The multiplier (111) multiplies the input (N)-bit data and (M)-bit data. The operation value of the multiplier (111) is output as (N+M)-bit data. Here, N and M are integers greater than 0. The first input section receiving (N)-bit data can be configured to receive variable parameters, and the second input section receiving (M)-bit data can be configured to receive constant parameters. However, the input data of the multiplier (111) is not limited to specific parameters. For example, weight parameters that have completed learning in a neural network model can be said to have constant characteristics based on processing elements. For example, input values ​​that are computed together with weights in a neural network model (e.g., activation parameters, feature map parameters, attention parameters, KV cache parameters, etc.) can be said to have variable characteristics based on processing elements.

[0163] That is, according to the examples of the present disclosure, since the input data of the processing element can operate by understanding the characteristics of constant values ​​and variable values, the computational efficiency of the neural processing unit (100) can be improved. However, the neural processing unit (100) is not limited to the characteristics of constant values ​​and variable values ​​of the input data.

[0164] Here, the meaning of a value possessing variable-like characteristics, or the meaning of a variable, implies that in the case of the value at the memory address where it is stored, it is updated whenever the incoming input data is updated. For example, the node data of each layer may be a MAC operation value reflecting the weight data of a neural network model; when using such a neural network model to infer object recognition from video data, the node data of each layer changes because the input image changes with every frame.

[0165] Here, the meaning of a value having constant-like characteristics, or the meaning of a constant, implies that the value at the memory address where it is stored is preserved regardless of updates to incoming input data. For example, as the weight data of a network serves as the unique inference criteria for a neural network model, the network's weight data may not change even if the model infers object recognition from video data.

[0166] That is, the multiplier (111) can be configured to receive one variable and one constant as input. To elaborate, the variable value input to the first input may be node data of a layer of the neural network model, and the node data may be input data of the input layer of the neural network model, the accumulated value of the hidden layer, and the accumulated value of the output layer. The constant value input to the second input may be weight data of the connection network of the neural network model.

[0167] The NPU controller (130) can be configured to improve memory reuse rate by taking into account the characteristics of constant values.

[0168] The variable value is the operation value of each layer, and the NPU controller (130) can recognize the reusable variable value based on the machine code of the compiled neural network model and control the NPU internal memory (120) to reuse memory.

[0169] The constant value is the weight data of each network, and the NPU controller (130) can recognize the constant value of a network that is repeatedly used based on the structural data of the neural network model or the neural network data locality information, and control the NPU internal memory (120) to reuse the parameter stored in the NPU internal memory (120). Reusing the parameter means that the parameter stored in the NPU internal memory (120) is not deleted or copied or moved to the dedicated memory (3000), but is reused in the next operation. According to the above configuration, as illustrated in FIG. 4, there is an effect of preventing power consumption due to the operation of the main memory (3000). In addition, there is an effect of eliminating the delay time that occurs when the neural processing unit (100) transmits data to the main memory (3000).

[0170] That is, the NPU controller (130) can determine reusable variable values ​​and reusable constant values ​​based on the machine code of the compiled neural network model. Accordingly, the NPU controller (130) can be configured to control the NPU internal memory (120) to reuse parameters stored in memory.

[0171] The processing element can restrict the operation of the multiplier (111) so that it does not perform operations because it knows that when 0 is input to one of the first and second inputs of the multiplier (111), the result of the operation becomes 0 even without performing operations.

[0172] For example, when 0 is input to one of the first input and second input sections of the multiplier (111), the multiplier (111) may be configured to operate in a zero skipping manner.

[0173] For zero skipping, each processing element (PE) included in the plurality of processing elements (400) can be enabled or disabled. The NPU controller (130) can be configured to provide an enable or disable signal to each processing element (PE) in clock units. Here, when a processing element (PE) is disabled, the multiplier (111) can be configured to be disabled according to the level of the first enable signal (En1). Accordingly, the power consumed in the operation of the multiplier (111) can be reduced. For example, information regarding the power consumption of the multiplier can be referenced in FIG. 4.

[0174] For zero skipping, each processing element (PE) included in the plurality of processing elements (400) can be enabled or disabled. The NPU controller (130) can be configured to provide an enable or disable signal to each processing element (PE) in clock units. Here, when a processing element (PE) is disabled, the adder (112) can be configured to be disabled according to the level of the second enable signal (En2). Accordingly, the power consumed in the operation of the adder (112) can be reduced. For example, information regarding the power consumption of the adder can be referenced in FIG. 4. In some examples, each processing element (PE) can be designed to receive a control signal from the NPU controller (130) for zero skipping operation control (i.e., enable or disable).

[0175] In some examples, each multiplier (111) of each processing element (PE) may be designed to receive each control signal for zero-skipping operation control from the NPU controller (130). According to the above configuration, the power consumption of the multiplier can be reduced by zero-skipping.

[0176] In some examples, each adder (112) of each processing element (PE) may be designed to receive each control signal for zero-skipping operation control from the NPU controller (130). According to the above configuration, the power consumption of the adder can be reduced by zero-skipping.

[0177] In some examples, each multiplier (111) and adder (112) of each processing element (PE) may be designed to simultaneously receive each control signal for zero-skipping operation control from the NPU controller (130). According to the above configuration, the power consumption of the multiplier and adder can be reduced by zero-skipping.

[0178] In some examples, the weights are constant values ​​that have been learned, and the machine code compiled for the neural network model containing said weights can be programmed to input each control signal for zero-skipping operation control to each processing element (PE) into which values ​​with zero weights are input.

[0179] The number of bits of data input to the first input section and the second input section can be determined according to the quantization of the node data and weight data of each layer of the neural network model. For example, the node data of the first layer may be quantized to 5 bits and the weight data of the first layer may be quantized to 7 bits. In this case, the first input section may be configured to receive 5-bit data, and the second input section may be configured to receive 7-bit data. That is, the number of bits of data input to each input section may be different from each other.

[0180] The processing element (PE) can be configured to receive quantization information of the data input to each input. The neural network data locality information may include quantization information of the input data and output data of the processing element (PE).

[0181] The neural processing unit (100) can control the quantized bit count to be converted in real time when quantized data stored in the internal memory (120) of the NPU is input to the inputs of the processing element. That is, the quantized bit count may differ for each layer, and the processing element may be configured to generate input data by receiving bit count information in real time from the neural processing unit (100) when the bit count of the input data is converted and converting the bit count in real time.

[0182] The accumulator (113) uses the adder (112) as many times as (L) loops to accumulate the operation value of the multiplier (111) and the operation value of the accumulator (113). Accordingly, the number of bits of data in the output and input sections of the accumulator (113) can be output as (N+M+log2(L)) bits. Here, L is an integer greater than 0.

[0183] When accumulation is finished, the accumulator (113) can receive an initialization reset signal and initialize the data stored inside the accumulator (113) to 0. However, examples according to the present disclosure are not limited thereto.

[0184] The accumulator (113) is configured to store the accumulated value even when zero skipping is enabled in the corresponding processing element (PE). Therefore, subsequent values ​​can be accumulated even when zero skipping is enabled.

[0185] The bit quantization unit (114) can reduce the number of bits of data output from the accumulator (113). The bit quantization unit (114) can be controlled by the NPU controller (130). The number of bits of the quantized data can be output as (X) bits. Here, X is an integer greater than 0. According to the above configuration, a plurality of processing elements are configured to perform MAC operations, and the plurality of processing elements have the effect of quantizing and outputting the MAC operation results. In particular, this quantization has the effect of further reducing power consumption as (L) loops increase. In addition, reducing power consumption has the effect of reducing heat generation of the edge device. In particular, reducing heat generation has the effect of reducing the possibility of malfunction caused by high temperature of the neural processing unit (100).

[0186] The output data (X) bit of the bit quantization unit (114) can be the node data of the next layer or the input data of the convolution. If the neural network model is quantized, the bit quantization unit (114) can be configured to receive quantized information from the neural network model. However, it is not limited to this, and it is also possible for the NPU controller (130) to be configured to analyze the neural network model and extract quantized information. Accordingly, the output data (X) bit can be converted into a number of quantized bits and output so as to correspond to the size of the quantized data. The output data (X) bit of the bit quantization unit (114) can be stored in the NPU internal memory (120) as a number of quantized bits.

[0187] A plurality of processing elements of a neural processing unit (100) according to one example of the present disclosure may include a multiplier (111), an adder (112), an accumulator (113), and a bit quantization unit (114). The plurality of processing elements can reduce data with a bit count of (N+M+log2(L)) bits output from the accumulator (113) to a bit count of (X) bits by the bit quantization unit (114). An NPU controller (130) can control the bit quantization unit (114) to reduce the bit count of the output data by a predetermined amount of bits from the least significant bit (LSB) to the most significant bit (MSB). When the bit count of the output data is reduced, power consumption, computational amount, and memory usage can be reduced. However, if the bit count is reduced to a length below a certain length, a problem may arise in which the inference accuracy of the neural network model may rapidly decrease. Therefore, the reduction in the number of bits of the output data, i.e., the quantization level, can be determined by comparing the reduction in power consumption, computational load, and memory usage relative to the reduction in the inference accuracy of the neural network model. The quantization level can also be determined by determining the target inference accuracy of the neural network model and testing it while gradually reducing the number of bits. The quantization level can be determined for each layer of the neural network model.

[0188] According to the processing element (PE) described above, by adjusting the number of bits of the (N)-bit data and (M)-bit data of the multiplier (111) and reducing the number of bits of the operation value (X)-bit by the bit quantization unit (114), the multiple processing elements have the effect of reducing power consumption while improving the MAC operation speed, and also have the effect of making the convolution operation of the neural network model more efficient.

[0189] FIG. 4b is a schematic conceptual diagram illustrating an SFU that can be applied to one example of the present disclosure.

[0190] Referring to FIG. 4b, the SFU (150) may include several function units. Each function unit may be operated optionally. Each function unit may be turned on or turned off optionally. That is, each function unit is configurable.

[0191] In other words, the SFU (150) may include various circuit units required for artificial neural network inference operations.

[0192] For example, the circuit units of the SFU (150) may include a function unit for skip-connection operation, a function unit for activation function operation, a function unit for pooling operation, a function unit for quantization / dequantization operation, a function unit for non-maximum suppression (NMS) operation, a function unit for softmax function operation, a function unit for batch-normalization operation, a function unit for interpolation operation, a function unit for concatenation operation, and a function unit for bias operation.

[0193] The functional units of the SFU (150) can be selectively turned on or off based on the data locality information of the neural network model. The data locality information of the neural network model may include control information related to the turning off or turning off of the corresponding functional unit when an operation for a specific layer is performed.

[0194] Among the functional units of the SFU (150), the activated units can be turned on. In this way, when some functional units of the SFU (150) are selectively turned off, the power consumption of the neural processing unit (100) can be reduced. Meanwhile, power gating may be used to turn off some functional units. Alternatively, clock gating may be performed to turn off some functional units.

[0195] FIG. 5 is an illustrative diagram showing a variation of the neural processing unit (100) illustrated in FIG. 3.

[0196] The neural processing unit (100) illustrated in FIG. 5 is substantially identical to the processing unit (100) illustrated in FIG. 3, except for a plurality of processing elements (110), so redundant descriptions may be omitted below merely for convenience of explanation.

[0197] The plurality of processing elements (110) illustrated exemplarily in FIG. 5 may further include, in addition to the plurality of processing elements (PE1 to PE12), respective register files (RF1 to RF12) corresponding to each of the processing elements (PE1 to PE12).

[0198] The plurality of processing elements (PE1 to PE12) and plurality of register files (RF1 to RF12) illustrated in FIG. 5 are merely examples for convenience of explanation, and the number of the plurality of processing elements (PE1 to PE12) and plurality of register files (RF1 to RF12) is not limited.

[0199] The size or number of multiple processing elements (110) can be determined by the number of multiple processing elements (PE1 to PE12) and multiple register files (RF1 to RF12). The size of the multiple processing elements (110) and multiple register files (RF1 to RF12) can be implemented in the form of an N x M matrix. Here, N and M are integers greater than 0.

[0200] The array size of multiple processing elements (110) can be designed by taking into account the characteristics of the neural network model in which the neural processing unit (100) operates. To elaborate, the memory size of the register file can be determined by taking into account the data size of the neural network model to be operated, the required operating speed, the required power consumption, etc.

[0201] The register files (RF1 to RF12) of the neural processing unit (100) are static memory units directly connected to the processing elements (PE1 to PE12). The register files (RF1 to RF12) may be composed of, for example, flip-flops and / or latches. The register files (RF1 to RF12) may be configured to store the MAC operation values ​​of the corresponding processing elements (PE1 to PE12). The register files (RF1 to RF12) may be configured to provide or receive weight data and / or node data with the NPU internal memory (120).

[0202] The register files (RF1 to RF12) can also be configured to function as temporary memory for the accumulator during MAC operations.

[0203] The neural processing unit (100) according to the examples of the present disclosure may be referred to as an AI semiconductor chip, an AI accelerator, a neural processor chip, etc.

[0204] Figure 6 is a block diagram showing a system according to one example.

[0205] Referring to FIG. 6, the system (1000) may include one or more NPUs (100-1, 100-2) mounted on a substrate, one or more memories (200-1, 200-2), a memory controller (250), one or more CPUs (300-1, 300-2), one or more GPUs (graphic processing units) (400), one or more MCUs (Micro Controller Units) (500), a controller (600), a system bus (800), and an I / O (input output) interface (900). Here, the NPU and CPU may be integrated into a single semiconductor chip or implemented as chiplets.

[0206] However, it can be seen that certain circuits among the components of the above system (1000) may be selectively excluded according to the operation scenario of a specific application where a specific function is unnecessary.

[0207] The system bus (800) may be implemented by an electrically conductive pattern formed on the substrate or semiconductor die. The system bus (800) enables high-speed communication. For example, the one or more NPUs (100-1, 100-2), the one or more memories (200-1, 200-2), the memory controller (250), the one or more CPUs (300-1, 300-2), one or more GPUs (400), and one or more MCUs (500) may communicate through the system bus (800).

[0208] The one or more NPUs (100-1, 100-2), the one or more memories (200-1, 200-2), the memory controller (250), the one or more CPUs (300-1, 300-2), one or more GPUs (400), and one or more MCUs (500) may be semiconductors implemented as electrical / electronic circuits. That is, the one or more NPUs (100-1, 100-2), the one or more memories (200-1, 200-2), the memory controller (250), the one or more CPUs (300-1, 300-2), one or more GPUs (400), and one or more MCUs (500) may be semiconductor circuits in which numerous electronic components (e.g., transistors, capacitors) are connected.

[0209] The one or more NPUs (100-1, 100-2), the one or more memories (200-1, 200-2), the memory controller (250), the one or more CPUs (300-1, 300-2), one or more GPUs (400), and one or more MCUs (500) make a request to the memory controller (250) through the system bus (500). Accordingly, the memory controller (250) can read or write data from at least one of the multiple memories (200-1, 200-2).

[0210] Among the above plurality of NPUs, the first NPU (100-1) may be a Type A NPU, and the second NPU (100-2) may be a Type B NPU.

[0211] And among the plurality of CPUs above, the first CPU (100-1) may be a CPU based on X86 architecture, and the second CPU (300-2) may be a CPU based on ARM architecture. According to another example, the first CPU (300-1) may be a CPU based on ARM architecture, and the second CPU (300-2) may be a CPU based on X86 architecture or a CPU based on RISC-V architecture.

[0212] Meanwhile, the above system (1000) may be implemented in the form of a System on Chip (SoC). An SoC refers to a semiconductor that contains the entire system on a single chip, and signifies a technology in which major semiconductor components, such as computational memory data conversion devices, are implemented on a single chip. A SiP refers to a semiconductor that contains the entire system in a single package, and signifies a technology in which major semiconductor components, such as computational memory data conversion devices, are implemented on a single package.

[0213] A neural processing unit (100) according to the examples of the present disclosure may be referred to as an AI semiconductor, an AI accelerator, a neural processor chip, etc. A system (1000) including a neural processing unit (100) according to the examples of the present disclosure may be referred to as an SoC, an AI processor package, an AI semiconductor, an AI accelerator, a neural processor chip, etc.

[0214] Semiconductor Package

[0215] Generally, semiconductor chips integrate numerous microcircuits, but they cannot function as finished semiconductor products on their own and are susceptible to damage from external physical or chemical shocks. Therefore, instead of using the chips as they are, they are packaged and used in electronic devices and other applications.

[0216] The reason semiconductor packaging is necessary is that, from the perspective of electrical connection, there is a difference in circuit width between semiconductor chips and the mainboards of electronic devices. Specifically, in the case of semiconductor chips, the size of connection pads and the spacing between them are very fine, whereas in the case of mainboards used in electronic devices, the size of component mounting pads and the spacing between them are much larger than the scale of semiconductor chips. Therefore, it is difficult to mount semiconductor chips directly onto such mainboards, and packaging technology is required to buffer the difference in circuit width between them.

[0217] Semiconductor packages can be broadly classified into conventional packages, where the wafer is cut into chip units for the packaging process, and wafer-level packages, where part or all of the packaging process is carried out at the wafer level and the wafer is cut into individual components later.

[0218] Conventional packages can be classified into ceramic packages and plastic packages depending on the packaging material. Plastic packages electrically connect cut chips by attaching them, and depending on the type of substrate serving as the medium, they can be further classified into leadframe type packages using leadframes and substrate type packages using substrates.

[0219] Wafer-level packaging refers to a package in which the packaging process is carried out at the wafer level. Wafer-level packaging can be classified into Re-Distribution Layer (RDL), which rearranges pads electrically connected to the outside on the chip via wafer-level processes; Flip Chip packaging, which performs the packaging process by forming solder bumps on the wafer; Wafer Level Chip Scale Package (WLCSP), which completes the package by forming wiring and solder balls directly on the wafer without an intermediary such as a substrate; and Through-Si Via (TSV) packaging, which provides internal connections between stacked chips using TSVs. Furthermore, WLCSP can be further classified into Fan-in WLCSP, which attaches wiring and solder balls directly onto the wafer, and Fan-out WLCSP, which rearranges the chip to create a molded wafer and forms wiring and attaches solder balls via wafer-level processes on a package larger than the chip size.

[0220] "Fan" refers to the chip size. A fan-in WLCSP is one where the package solder balls are fully implemented within the chip size, while a fan-out WLCSP is one where the package solder balls are implemented outside the fan as well.

[0221] In summary, a package according to one example of the present disclosure may include at least one memory chip and at least one neural processor chip. The package may be implemented as a conventional package or a wafer-level package. The conventional package may be implemented as a ceramic package or a plastic package. The wafer-level package may be implemented as a flip-chip package, a WLCSP package, or a TSV package.

[0222] Below, we will examine fan-in semiconductor packages and fan-out semiconductor packages in more detail with reference to the drawings.

[0223] I. Fan-in Semiconductor Package

[0224] Fan-in WLCSP is a package in which wiring, insulation layers, and solder balls are formed directly on the wafer.

[0225] FIGS. 7A and 7B are cross-sectional views schematically showing the pre- and post-packaging of a fan-in semiconductor package, and FIG. 8 is a cross-sectional view schematically showing the packaging process of a fan-in semiconductor package.

[0226] Referring to the drawing, the semiconductor chip (2220) may be, for example, a bare integrated circuit (IC) comprising a body (2221) containing at least some of silicon dioxide (SiO2), silicon (Si), germanium (Ge), and gallium arsenide (GaAs), a connection pad (2222) formed on one side of the body (2221) containing at least some of a conductive material such as copper (Cu), titanium (Ti), and aluminum (Al), and a passivation film (2223), such as an oxide film or a nitride film, formed on one side of the body (2221) and covering at least a portion of the connection pad (2222). At this time, since the connection pad (2222) is very small, the integrated circuit (IC) is difficult to mount not only on the main board of an electronic device but also on an intermediate-level printed circuit board (PCB).

[0227] Accordingly, in order to rewire the connection pad (2222), a connecting member (2240) is formed on the semiconductor chip (2220) to match the size of the semiconductor chip (2220). The connecting member (2240) can be formed by forming an insulating layer (2241) on the semiconductor chip (2220) with an insulating material such as a photosensitive insulating resin (PID), forming a via hole (2243) that opens the connection pad (2222), and then forming a wiring pattern (2242) and a via (2243). After that, a passivation layer (2250) that protects the connecting member (2240) is formed, an opening (2251) is formed, and then an under-bump pad (2260), etc., is formed. That is, through a series of processes, for example, a fan-in semiconductor package (2200) including a semiconductor chip (2220), a connecting member (2240), a passivation layer (2250), and an under-bump pad (2260) is manufactured.

[0228] To elaborate, semiconductor chips must be connected to the main board, but it is impossible to manufacture the circuits on the main board as finely as the semiconductors. While the spacing between terminals on a semiconductor chip is approximately 100 µm (micrometers) thick, the spacing between terminals on a main board is about 350 µm, which can be about four times greater. Therefore, a bridge is required to connect the semiconductor chip and the main board, and this is the package board. The package board connects the semiconductor terminals one by one to the main board, which has a wide linewidth; this process is called 'rewiring'.

[0229] As such, a fan-in semiconductor package is a package type in which all connection pads of a semiconductor chip, such as I / O (Input / Output) terminals, are placed inside the device; fan-in semiconductor packages have good electrical characteristics and can be produced at a low cost.

[0230] The fan-in semiconductor package form factor has the advantage of enabling fast signal transmission while being compact.

[0231] However, fan-in semiconductor packages have many spatial constraints because all I / O terminals must be placed inside the semiconductor chip. Therefore, this structure is difficult to apply to semiconductor chips with a large number of I / O terminals or small semiconductor chips. Even if the size and spacing of the semiconductor chip's I / O terminals are enlarged through a rewiring process, it may be difficult to achieve a size and spacing sufficient for direct mounting on the mainboard of an electronic device.

[0232] FIG. 9A is a schematic cross-sectional view showing a case where a fan-in semiconductor package is mounted on an interposer substrate and finally mounted on a main board of an electronic device, and FIG. 9B is a schematic cross-sectional view showing a case where a fan-in semiconductor package is embedded within an interposer substrate and finally mounted on a main board of an electronic device.

[0233] Referring to the drawing, the underbump pads (2260) of the semiconductor chip (2220), i.e., the I / O terminals, of the fan-in semiconductor package (2200) are rewired once again through the interposer substrate (2301), and finally, the fan-in semiconductor package (2200) can be mounted on the main board (2500) of an electronic device with the package mounted on the interposer substrate (2301). At this time, the solder ball (2270), etc., can be fixed with an underfill resin (2280), etc., and the outer side can be covered with a sealing material (2290), etc. Alternatively, the fan-in semiconductor package (2200) may be embedded within a separate interposer substrate (2302), and in the embedded state, the connection pads (2222), i.e., I / O terminals of the semiconductor chip (2220), are rewired once again by the interposer substrate (2302) and finally mounted on the main board (2500) of the electronic device.

[0234] As such, since it is difficult to mount a fan-in semiconductor package directly onto the mainboard of an electronic device, it can be mounted on a separate interposer substrate and then mounted onto the mainboard of the electronic device after undergoing a packaging process, or it can be mounted onto the mainboard of the electronic device while embedded within the interposer substrate.

[0235] A neural processing unit (100) according to one example of the present disclosure may be implemented in a fan-in semiconductor package.

[0236] A package according to various examples of the present disclosure may be referred to as an AI processor package by including a neural processing unit (100).

[0237] II. Fan-Out Semiconductor Package

[0238] Figure 10 is a cross-sectional view showing a schematic appearance of a fan-out semiconductor package.

[0239] Referring to the drawing, the fan-out semiconductor package (2100) is, for example, protected on the outside of the semiconductor chip (2120) by a sealing material (2130), and the connection pad (2122) of the semiconductor chip (2120) is rewired to the outside of the semiconductor chip (2120) by a connecting member (2140). At this time, a passivation layer (2202) may be further formed on the connecting member (2140), and an under-bump pad (2160) may be further formed at the opening of the passivation layer (2202). A solder ball (2170) may be further formed on the under-bump pad (2160). The semiconductor chip (2120) may be an integrated circuit (IC) including a body (2121), a connection pad (2122), a passivation film (not shown), etc. The connecting member (2140) may include an insulating layer (2141), a redistribution layer (2142) formed on the insulating layer (2241), and a via (2143) that electrically connects the connection pad (2122) and the redistribution layer (2142).

[0240] In such a manufacturing process, a connecting member (2140) can be formed after a sealing material (2130) is formed on the outer side of the semiconductor chip (2120). In this case, since the connecting member (2140) is formed from a via and a redistribution layer that connects to the connection pad (2122) of the semiconductor chip (2120), the via (2143) can be formed to have a smaller width closer to the semiconductor chip (see enlarged area).

[0241] As such, a fan-out semiconductor package is a form in which I / O terminals are rewired and arranged to the outside of the semiconductor chip through a connecting member formed on the semiconductor chip. As described above, a fan-in semiconductor package requires all I / O terminals of the semiconductor chip to be placed inside the semiconductor chip, and since the ball size and pitch must be reduced when the device size is reduced, a standardized ball layout cannot be used. On the other hand, since a fan-out semiconductor package is a form in which I / O terminals are rewired and arranged to the outside of the semiconductor chip through a connecting member formed on the semiconductor chip, a standardized ball layout can be used as is even if the size of the semiconductor chip is reduced, and as described below, it can be mounted on the main board of an electronic device without a separate interposer board.

[0242] FIG. 11 is a schematic cross-sectional view showing a case where a fan-out semiconductor package is mounted on a main board of an electronic device.

[0243] Referring to the drawing, the fan-out semiconductor package (2100) can be mounted on the main board (2500) of an electronic device through solder balls (2170), etc. That is, as described above, the fan-out semiconductor package (2100) forms a connecting member (2140) that can rewire the connection pad (2122) to a fan-out area extending beyond the size of the semiconductor chip (2120) on the semiconductor chip (2120), so a standardized ball layout can be used as is, and as a result, it can be mounted on the main board (2500) of an electronic device without a separate interposer board, etc.

[0244] As such, since fan-out semiconductor packages can be mounted on the mainboard of electronic devices without a separate interposer substrate, they can be made thinner compared to fan-in semiconductor packages that use an interposer substrate, enabling miniaturization and thinning.

[0245] In addition, it has excellent thermal and electrical properties, making it particularly suitable for mobile products. Furthermore, it can be implemented more compactly than the general POP (Package on Package) type using a printed circuit board (PCB), and can improve problems caused by warping.

[0246] Meanwhile, a fan-out semiconductor package refers to a packaging technology designed to mount semiconductor chips on the mainboard of electronic devices and to protect them from external shocks; it differs in scale and application from the fan-in semiconductor package and is a different concept from printed circuit boards (PCBs), such as interposer substrates, in which the fan-in semiconductor package is embedded.

[0247] A neural processing unit (100) according to one example of the present disclosure may be implemented in a fan-out semiconductor package.

[0248] A package according to various examples of the present disclosure may be referred to as an AI processor package by including a neural processing unit (100).

[0249] III. Package on Package (PoP)

[0250] FIG. 12 is a cross-sectional view showing a package-on-package (PoP) according to the first example.

[0251] Referring to FIG. 12, a package-on-package (PoP) (3000) can be formed by attaching a second package (3200) to a first package (3100).

[0252] The first package (3100) may be a semiconductor package in the form of a Fan Out Wafer Level Package (FOWLP), for example. The second package (3200) may also be in the form of a Fan Out Wafer Level Package (FOWLP) or a Fan In Package.

[0253] The first package (3100) may include a first semiconductor chip (3110).

[0254] The substrate forming the first semiconductor chip (3110) may be a semiconductor substrate or a glass substrate.

[0255] A semiconductor substrate may have an active surface and an inactive surface opposite to the active surface. A semiconductor device comprising a plurality of individual devices of various types may be formed on the active surface of the first semiconductor chip (3110).

[0256] The above glass substrate may include silica (SiO2) and boric acid (B2O3). In this case, physical and electrical properties can be optimized by controlling the content of silica (SiO2) and boric acid (B2O3), and specific properties can be enhanced through additional metal oxide doping. Additionally, the above glass substrate may include alumina (Al2O3). Silica (SiO2) may be included in a content of 60–80%, enabling the achievement of high heat resistance and mechanical strength, and reducing dielectric loss. Boric acid (B2O3) may be included in a content of 10–25%, enabling a reduction in the coefficient of thermal expansion, improvement in processability, provision of a high dielectric constant, and supplementation of mechanical strength. Alumina (Al2O3) may be included in a content of 0–10%, not only enhancing chemical durability and improving mechanical strength but also increasing the dielectric constant. The metal oxide for property control is titania (TiO2), It may include strontium oxide (SrO), magnesia (MgO), and calcium oxide (CaO). Titania (TiO2) may be included in an amount of 1 to 5%, which can increase the dielectric constant while simultaneously maintaining mechanical strength. Strontium oxide (SrO) may be included in an amount of 1 to 5%, which improves high-frequency signal characteristics and enables low dielectric loss. Magnesia (MgO) may be included in an amount of 0 to 5%, which can reduce dielectric loss and increase heat resistance. Calcium oxide (CaO) may be included in an amount of 0 to 3%, which can reduce dielectric loss and improve thermal stability.

[0257] The first semiconductor chip (3110) may include a plurality of pads (3110p). The plurality of pads (3110p) may be electrically connected to the semiconductor device included in the first semiconductor chip (3110). The first semiconductor chip (3110) may be a single semiconductor chip, but is not limited thereto. For example, the first semiconductor chip (3110) may be a stack of a plurality of semiconductor chips.

[0258] In exemplary embodiments, the first semiconductor chip (3110) may be, for example, a memory semiconductor chip. The memory semiconductor chip may be, for example, a Static Random Access Memory (SRAM).

[0259] Alternatively, in exemplary embodiments, the first semiconductor chip (3110) may be a logic chip. For example, the first semiconductor chip (3110) may be a Neural Processing Unit (NPU), an Image Signal Processor (ISP), a Central Processor Unit (CPU), a Micro Processor Unit (MPU), a Graphics Processing Unit (GPU), or an Application Processor (AP).

[0260] The first package (3100) may include a first sealing layer (3120) covering at least a portion of the first semiconductor chip (3110). The first sealing layer (3120) may cover the side of the first semiconductor chip (3110) and the lower surface of the first semiconductor chip (3110) on which a pad (3110p) is provided. The first sealing layer (3120) may have an opening for exposing the pad (3110p) of the first semiconductor chip (3110).

[0261] The first sealing layer (3120) may include an insulating material.

[0262] The first sealing layer (3120) may include a through hole (3120H) that penetrates vertically through the first sealing layer (3120). The through hole (3120H) may be provided in a peripheral portion of the first semiconductor chip (3110).

[0263] The first package (3100) may include a first redistribution structure (3130, 3140) provided on the first semiconductor chip (3110). The first redistribution structure (3130, 3140) may include a first redistribution pattern (3130) and a first insulation pattern (3140).

[0264] The first rewiring pattern (3130) can electrically connect the pad (3110p) of the first semiconductor chip (3110) to the external connection terminal (3190). Additionally, the first rewiring pattern (3130) can be electrically connected to the second rewiring pattern (3230). Through the first rewiring pattern (3130) and the second rewiring pattern (3230), the first semiconductor chip (3110) and the second semiconductor chip (3210) can be electrically connected, and the second semiconductor chip (3210) can be electrically connected to the external connection terminal (1390).

[0265] More specifically, the first redistribution pattern (3130) may be composed of a plurality of sub-redistribution patterns, and the sub-redistribution patterns may have a multilayer structure. For example, the first redistribution pattern (3130) may include a first sub-redistribution pattern (3131) and a second sub-redistribution pattern (3133). The first sub-redistribution pattern (3131) is formed on the first sealing layer (3120) and may be connected to a pad (3110p) of the first semiconductor chip (3110). A portion of the first sub-redistribution pattern (3131) may be connected to the second redistribution pattern (3230) through the first sealing layer (3120) and the second insulation pattern (3240). The second sub-rewiring pattern (3133) extends over the first insulation pattern (3140) and can be connected to the first sub-rewiring pattern (3131) through the first insulation pattern (3140).

[0266] The first insulation pattern (3140) may be provided on the lower surface of the first sealing layer (3120). The first insulation pattern (3140) may cover the first sub-rewiring pattern (3131) and have an opening that exposes a portion of the first sub-rewiring pattern (3131).

[0267] A protective layer (3150) may be formed on the first insulating pattern (3140). The protective layer (3150) may expose a portion of the second sub-rewiring pattern (3133). An external connection terminal (3190) may be disposed on the portion of the second sub-rewiring pattern (3133) exposed by the protective layer (3150). The external connection terminal (3190) may be, for example, a solder ball, a bump, a copper filler bump, or a copper-to-copper direct bonding. The external connection terminal (3190) may electrically connect the package-on-package (PoP) (3000) and an external device.

[0268] The second package (3200) may be placed on the first package (3100). The second package (3200) may include a second semiconductor chip (3210). The second semiconductor chip (3210) may include a pad (3210p). The second semiconductor chip (3210) may be a single semiconductor chip, but is not limited thereto. The second semiconductor chip (3210) may be at least one semiconductor chip. For example, the second semiconductor chip (210) may be a stack of multiple semiconductor chips.

[0269] In exemplary embodiments, the second semiconductor chip (3210) may be, for example, at least one memory semiconductor chip. Or, in exemplary embodiments, the second semiconductor chip (3210) may be a logic chip.

[0270] The second package (3200) may include a second sealing layer (3220) that covers at least a portion of the second semiconductor chip (3210). For example, the second sealing layer (3220) may cover the side of the second semiconductor chip (3210) and cover the lower surface of the second semiconductor chip (3210) on which the pad (3210p) is provided. The second sealing layer (3220) may have an opening for exposing the pad (3210p) of the second semiconductor chip (210).

[0271] At this time, the second sealing layer (3220) may not cover the upper surface of the second semiconductor chip (3210) opposite to the lower surface of the second semiconductor chip (3210).

[0272] The second sealing layer (3220) may include an insulating material. In exemplary embodiments, the second sealing layer (3220) may include a photosensitive material. For example, the second sealing layer (3220) may be composed of a polymer material such as polyimide. However, the material constituting the second sealing layer (3220) is not limited thereto, and, for example, the second sealing layer (3220) may include EMC (Epoxy Molding Compound).

[0273] The second package (3200) may include a second redistribution structure (3230, 3240) provided between the second sealing layer (3220) and the first sealing layer (120). The second redistribution structure (3230, 3240) may include a second redistribution pattern (3230) and a second insulation pattern (3240). The second redistribution pattern (3230) may extend along the surface of the second sealing layer (3220) and may be electrically connected to a pad (3210p) of the second semiconductor chip (3210).

[0274] In the embodiments, the electrical connection between the second package (3200) and the first package (3100) can be made through the connection of the first redistribution pattern (3130) and the second redistribution pattern (3230).

[0275] Since the package-on-package (PoP) (3000) does not include inter-package connection terminals, such as solder balls for connecting the second package (3200) and the first package (3100), the semiconductor package manufacturing process can be simplified and a thinner PoP-type semiconductor package can be manufactured.

[0276] Generally, in the case of a semiconductor package in the form of a PoP in which multiple packages are stacked, there was a problem in which damage such as cracks occurred at the connection terminals between packages due to warpage of the semiconductor packages, thereby degrading the reliability of the semiconductor packages. However, according to exemplary embodiments, the second package (3200) and the first package (3100) can be electrically connected without the connection terminals between packages (e.g., solder balls) that are vulnerable to warpage, so the reliability of the semiconductor packages can be further improved.

[0277] According to one example of the present disclosure, a second package (3200) includes a neural processing unit (100), and a first package (3100) may be configured to include one or a plurality of memory chips.

[0278] According to one example of the present disclosure, a first package (3100) may include a neural processing unit (100), and a second package (3200) may be configured to include one or a plurality of memory chips. Here, if the neural processing unit (100) generates more heat than the memory, it may be advantageous for the neural processing unit (100) to be positioned above the memory for heat dissipation.

[0279] A package according to various examples of the present disclosure may be referred to as an AI processor package by including a neural processing unit (100).

[0280] In addition, the technical features of different examples of the present disclosure may be combined and modified.

[0281] FIGS. 13a to 13d are cross-sectional views illustrating a method for manufacturing a package-on-package (PoP) (3000) illustrated in FIG. 12.

[0282] Referring to FIG. 13a, a second semiconductor chip (3210) is placed on a carrier (3011), and a second sealing layer (3220) covering the second semiconductor chip (3210) is formed. The second sealing layer (3220) may be formed to cover the side of the second semiconductor chip (3210) and to cover the surface of the second semiconductor chip (3210) on which a pad (3210p) is provided. In exemplary embodiments, to form the second sealing layer (3220), an insulating film may be applied on the carrier (3011) and the second semiconductor chip (3210), and a portion of the insulating film may be removed so that the pad (3210p) of the second semiconductor chip (3210) is exposed. The insulating film may include, for example, a photosensitive material.

[0283] After forming the second sealing layer (3220), a second redistribution structure (3230, 240) can be formed on the second sealing layer (3220) and the second semiconductor chip (3210). Specifically, a second redistribution pattern (3230) is formed on the upper part of the second sealing layer (3220) and on the pad (3210p) of the second semiconductor chip (3210). For example, the second redistribution pattern (3230) can be formed through a seed film formation process, a mask process, and an electroplating process. After forming the second redistribution pattern (3230), in order to form a second insulation pattern (3240), an insulating film can be formed on the second sealing layer (3220) and the second redistribution pattern (3230), and a part of the insulating film can be removed to form an opening (3240H) for exposing a part of the second redistribution pattern (3230).

[0284] Referring to FIG. 13b, a first semiconductor chip (3110) is placed on a second insulating pattern (3240). Between the first semiconductor chip (3110) and the second insulating pattern (3240), an adhesive layer (3119) for fixing the first semiconductor chip (3110) may be provided. The adhesive layer (3119) may include, for example, a die attach film. Additionally, the adhesive layer (3119) may include a material with high thermal conductivity so that heat from the first semiconductor chip (3110) is effectively dissipated.

[0285] After placing the first semiconductor chip (3110), a first sealing layer (3120) covering the first semiconductor chip (3110) can be formed. The first sealing layer (3120) may be formed to have an opening to expose the pad (3110p) of the first semiconductor chip (3110) and may be formed to have a through hole (3120H) penetrating the first sealing layer (3120) to expose the second redistribution pattern (3230). To form the first sealing layer (3120), an insulating film may be applied on the first semiconductor chip (3110), a portion of the insulating film may be removed so that the pad (3110p) of the first semiconductor chip (3110) is exposed, and a through hole (3120H) penetrating the insulating film vertically may be formed so that the second redistribution pattern (3230) is exposed.

[0286] In exemplary embodiments, the first sealing layer (3120) is formed through a lamination process using a polymer material such as polyimide and can cover the side of the first semiconductor chip (3110) and the surface of the first semiconductor chip (3110) on which the pad (3110p) is provided. In this case, compared to the step of forming a mold material covering the side of the first semiconductor chip (3110) and sequentially forming an insulating material on the lower surface of the first semiconductor chip (3110), the first sealing layer (3120) covering the side of the first semiconductor chip (3110) and the surface of the first semiconductor chip (3110) can be formed through a single lamination process, thereby simplifying the semiconductor package manufacturing process.

[0287] Referring to FIG. 13c, a first redistribution structure (3130, 3140) can be formed on a first sealing layer (3120) and a first semiconductor chip (3110). To form the first redistribution structure (3130, 3140), a first sub-redistribution pattern (3131), a first insulation pattern (3140), and a second sub-redistribution pattern (3133) can be formed sequentially. More specifically, the first sub-redistribution pattern (3131) is formed on the first sealing layer (3120), contacts a pad (3110p) of the first semiconductor chip (3110), and extends along a through hole (3120H) of the first sealing layer (3120) to contact the second redistribution pattern (3230). For example, the first sub-rewiring pattern (3131) can be formed through a seed film formation process, a mask process, and an electroplating process. After forming the first sub-rewiring pattern (3131), in order to form the first insulation pattern (3140), an insulating film can be formed on the first sealing layer (3120) and the first sub-rewiring pattern (3131), and a portion of the insulating film can be removed to form an opening to expose a portion of the first sub-rewiring pattern (3131). After forming the first insulation pattern (3140), a second sub-rewiring pattern (3133) is formed on the first insulation pattern (3140). The second sub-rewiring pattern (3133) can be formed to be connected to the first sub-rewiring pattern (3131) through the first insulation pattern (3140).

[0288] For example, the second sub-redistribution pattern (3133) can be formed through a seed film formation process, a mask process, and an electroplating process.

[0289] Subsequently, a protective layer (3150) is formed on the first insulating pattern (3140). The protective layer (3150) may be formed to have an opening that exposes a portion of the second sub-rewiring pattern (3133). After forming the protective layer (3150), an external connection terminal (3190) may be attached to the second sub-rewiring pattern (3133) exposed by the opening of the protective layer (3150). The external connection terminal (3190) may be, for example, a solder ball or a bump.

[0290] Referring to FIG. 13d, the carrier (3011 in FIG. 13c) is removed, and the semiconductor packages are individualized into individual semiconductor packages through a sawing process. That is, the semiconductor package shown in FIG. 13c can be cut along the scribe line (Fig. 13c, SL) and separated into a plurality of individual semiconductor packages.

[0291] FIG. 14 is a cross-sectional view showing a package-on-package (PoP) according to the second example.

[0292] In FIG. 14, the same description as previously explained is omitted or simplified.

[0293] Referring to FIG. 14, the semiconductor package-on-package (3000b) may include a first package (3100b) and a second package (3200b).

[0294] The first package (3100b) may be, for example, a semiconductor package in the form of FOWLP.

[0295] The first package (3100b) may include a plurality of semiconductor chips (3111, 3113).

[0296] For example, the first package (3100b) may include a first lower semiconductor chip (3111) and a second lower semiconductor chip (3113) that are horizontally spaced apart. In exemplary embodiments, the first lower semiconductor chip (3111) and the second lower semiconductor chip (3113) may be semiconductor chips of the same type. Or, in exemplary embodiments, the first lower semiconductor chip (3111) and the second lower semiconductor chip (3113) may be semiconductor chips of different types.

[0297] The first package (3100b) may include a first sealing layer (3120) covering at least a portion of the first lower semiconductor chip (3111) and at least a portion of the second lower semiconductor chip (3113). The first sealing layer (3120) may include an insulating material. In exemplary embodiments, the first sealing layer (3120) may be composed of a photosensitive material, such as a polymer material such as polyimide.

[0298] The second package (3200b) may include a plurality of semiconductor chips (3211, 3213).

[0299] For example, the second package (3200b) may include a first upper semiconductor chip (3211) and a second upper semiconductor chip (3213) that are horizontally spaced apart. In exemplary embodiments, the first upper semiconductor chip (3211) and the second upper semiconductor chip (3213) may be semiconductor chips of the same type. Or, in exemplary embodiments, the first upper semiconductor chip (3211) and the second upper semiconductor chip (3213) may be semiconductor chips of different types.

[0300] Furthermore, in exemplary embodiments, the second package (3200b) may be a semiconductor package in the form of a system in package in which various circuit elements, such as passive elements (3160), that perform functions such as signal processing are packaged. The passive elements (3160) may include resistors, capacitors, inductors, diodes, piezoelectric components, etc.

[0301] The second package (3200b) may include a second sealing layer (3220b) covering at least a portion of the first upper semiconductor chip (3211) and at least a portion of the second upper semiconductor chip (3213). The second sealing layer (3220b) may be filled between the first upper semiconductor chip (3211) and the second upper semiconductor chip (3213). In exemplary embodiments, the second sealing layer (3220b) may be made of EMC, but is not limited thereto.

[0302] According to one example of the present disclosure, a first package (3100b) of a semiconductor package-on-package (3000b) may include a neural processing unit (100), and a second package (3200b) may be configured to include one or a plurality of memory chips. Here, if the neural processing unit (100) generates more heat than the memory, it may be advantageous for the neural processing unit (100) to be positioned above the memory for heat dissipation.

[0303] In addition, the technical features of different examples of the present disclosure may be combined and modified.

[0304] FIGS. 15a to 15e describe a method for manufacturing a package-on-package (PoP) (3000b) illustrated in FIG. 14.

[0305] Referring to FIG. 15a, a first upper semiconductor chip (3211) and a second upper semiconductor chip (3213) are placed on a support substrate (3013). Subsequently, a second sealing layer (3220b) covering the first upper semiconductor chip (3211) and the second upper semiconductor chip (3213) is formed on the support substrate (3013). In exemplary embodiments, the second sealing layer (3220b) may include EMC. Alternatively, in exemplary embodiments, the second sealing layer (3220b) may include an insulating film containing a photosensitive material, such as the second sealing layer (3220) of FIG. 13a.

[0306] Referring to FIG. 15b, the support substrate (3013) is removed from the result of FIG. 15a, and the result is flipped over and placed on a carrier (3011).

[0307] Subsequently, a second redistribution structure (3230, 3240a) is formed on the second sealing layer (3220b), the first upper semiconductor chip (3211), and the second upper semiconductor chip (3213). To form the second redistribution structure (3230, 3240a), a first sub-insulating pattern (3241), a second redistribution pattern (3230), and a second sub-insulating pattern (3243) can be formed sequentially. More specifically, the first sub-insulating pattern (3241) can form an insulating film on the first upper semiconductor chip (3211) and the second upper semiconductor chip (3213), and form an opening to expose the pad (3211p) of the first upper semiconductor chip (3211) and the pad (3213p) of the second upper semiconductor chip (3213) by removing a portion of the insulating film. After forming the first sub-insulating pattern (3241), a second redistribution pattern (3230) is formed on the first sub-insulating pattern (3241). For example, the second redistribution pattern (3230) can be formed through a seed film formation process, a mask process, and an electroplating process. After forming the second redistribution pattern (3230), a second sub-insulating pattern (3243) covering the second redistribution pattern (3230) can be formed on the first sub-insulating pattern (3241). The second sub-insulating pattern (3243) can be formed to have an opening (3243H) for exposing a portion of the second redistribution pattern (3230).

[0308] Referring to FIG. 15c, a first lower semiconductor chip (3111) and a second lower semiconductor chip (3113) are placed on a second sub-insulating pattern (3243). An adhesive layer (3119) for fixing the first lower semiconductor chip (3111) and the second lower semiconductor chip (3113) may be provided between the first lower semiconductor chip (3111) and the second sub-insulating pattern (3243) and between the second lower semiconductor chip (3113) and the second sub-insulating pattern (3243).

[0309] After forming the first lower semiconductor chip (3111) and the second lower semiconductor chip (3113), a first sealing layer (3120) covering the first lower semiconductor chip (3111) and the second lower semiconductor chip (3113) can be formed. The first sealing layer (3120) can be formed to have a through hole (3120H) that vertically penetrates the first sealing layer (3120) and an opening that exposes the pad (3111p) of the first lower semiconductor chip (3111) and the pad (3113p) of the second lower semiconductor chip (3113) through a method similar to that described in FIG. 13b.

[0310] Referring to FIG. 15d, a first redistribution structure (3130, 3140) can be formed on a first sealing layer (3120), a first lower semiconductor chip (3111), and a second lower semiconductor chip (3113). To form the first redistribution structure (3130, 3140), a first sub-redistribution pattern (3131), a first insulation pattern (3140), and a second sub-redistribution pattern (3133) can be formed sequentially through a method similar to that described in FIG. 13c.

[0311] Afterwards, an external connection terminal (3190) attached to a second sub-rewiring pattern (3133) exposed by an opening in the protective layer (3150) can be formed through a method similar to that described in FIG. 13c.

[0312] Referring to FIG. 15e, the carrier (3011 in FIG. 15d) is removed, and the semiconductor package shown in FIG. 15d is cut along the scribe line (Fig. 15d, SL) to individualize the semiconductor package of FIG. 15d into a plurality of individual semiconductor packages.

[0313] FIG. 16 is a cross-sectional view showing a package-on-package (PoP) according to the third example.

[0314] FIG. 16 is a cross-sectional view showing a semiconductor package-on-package (3000c) according to exemplary embodiments of the present disclosure. In FIG. 16, descriptions identical to those previously described are omitted or simplified.

[0315] Referring to FIG. 16, the semiconductor package-on-package (3000c) may include a first package (3310), a second package (3320), a third package (3330), and a fourth package (3340) stacked in a vertical direction. The first package (3310), the second package (3320), the third package (3330), and the fourth package (3340) may each be semiconductor packages in the form of FOWLP.

[0316] The first package (3310) may include a first semiconductor chip (3311), a first sealing layer (3312), and a first rewiring structure (3313, 3314), the second package (3320) may include a second semiconductor chip (3321), a second sealing layer (3322), and a second rewiring structure (3323, 3324), and the third package (3330) may include a third semiconductor chip (3331), a third sealing layer (3332), and a third rewiring structure (3333, 3334).

[0317] For example, the first package (3310), the second package (3320), and the third package (3330) may have technical features similar to the first package (3100 of FIG. 12) described earlier with reference to FIG. 12, so a detailed description is omitted.

[0318] The fourth package (3340) may include a fourth semiconductor chip (3341), a fourth sealing layer (3342), and a fourth rewiring structure (3343, 3344). For example, since the fourth package (3340) may have technical features similar to the second package (3200 of FIG. 12) described earlier with reference to FIG. 12, a detailed description is omitted.

[0319] The first semiconductor chip (3311) of the first package (3310), the second semiconductor chip (3321) of the second package (3320), the third semiconductor chip (3331) of the third package (3330), and the fourth semiconductor chip (3341) of the fourth package (3340) may be semiconductor chips of the same type or semiconductor chips of different types.

[0320] In exemplary embodiments, electrical connections between the first to fourth semiconductor chips (3311, 3321, 3331, 3341) may be implemented through a first redistribution pattern (3313) of a first package (3310) extending through a first sealing layer (3312) and a second insulation pattern (3324), a second redistribution pattern (3323) of a second package (3320) extending through a second sealing layer (3322) and a third insulation pattern (3334), a third redistribution pattern (3333) of a third package (3330) extending through a third sealing layer (3332) and a fourth insulation pattern (3344), and a fourth redistribution pattern (3343) of a fourth package (3340).

[0321] The first to fourth packages (3310, 3320, 3330, 3340) can be electrically connected without inter-package connection terminals that are susceptible to bending, so the reliability of the semiconductor package can be further improved.

[0322] In addition, since multiple packages can be stacked without inter-package connection terminals, thinner semiconductor packages can be manufactured.

[0323] A semiconductor package-on-package (3000c) according to one example of the present disclosure may include a plurality of stacked packages. Here, the first package (3310) includes a neural processing unit (100), and the plurality of vertically stacked packages (e.g., second to fourth packages (3320, 3330, 3340)) may be stacked packages including one or more memory chips. Here, if the neural processing unit (100) generates more heat than the memory, it may be advantageous for the neural processing unit (100) to be positioned above the memory for heat dissipation.

[0324] In addition, the technical features of different examples of the present disclosure may be combined and modified.

[0325] FIG. 17 is a cross-sectional view showing a package-on-package (PoP) according to the fourth example.

[0326] In FIG. 17, the same descriptions as previously explained are omitted or simplified.

[0327] Referring to FIG. 17, a semiconductor package-on-package (3000d) according to the fourth example may include a first package (3310), a second package (3320), a third package (3330), and a fourth package (3340) stacked in a vertical direction.

[0328] For example, the first to fourth packages (3310, 3320, 3330, 3340) may have technical features similar to the semiconductor package-on-package (3000c) described above with reference to FIG. 16, so a detailed description is omitted.

[0329] The semiconductor package-on-package (3000d) may include an electromagnetic shielding layer (3350) covering at least a portion of the first package (3310), second package (3320), third package (3330), and fourth package (3340) stacked in a vertical direction.

[0330] For example, as illustrated, the electromagnetic shielding layer (3350) may cover the sidewall of the first package (3310), the sidewall of the second package (3320), the sidewall of the third package (3330), the sidewall of the fourth package (3340), and the top surface. By shielding electromagnetic interference (EMI), the electromagnetic shielding layer (3350) can reduce the performance degradation of the semiconductor package-on-package (3000d) caused by EMI.

[0331] The semiconductor package-on-package (3000d) may include an outer sealing layer (3360) covering an electromagnetic shielding layer (3350) covering the first to fourth packages (3310, 3320, 3330, 3340). In exemplary embodiments, the outer sealing layer (3360) may include a material with excellent thermal conductivity so as to improve the heat dissipation characteristics of the semiconductor package-on-package (3000d).

[0332] The semiconductor package-on-package (3000d) may include a lower conductive layer (3370) and a thermal conductive film (3380) provided on the lower surface of the first package (3310) and the lower surface of the outer sealing layer (3360).

[0333] The lower conductive layer (3370) may extend along the surface of the first insulating pattern (3314) of the first package (3310), the surface of the outer sealing layer (3360), and / or the surface of the electromagnetic shielding layer (3350) between the first package (3310) and the outer sealing layer (3360).

[0334] A portion of the lower conductive layer (3370) extends along the first insulating pattern (3314) and can be connected to the first redistribution pattern (3313) of the first package (3310) through an opening in the first insulating pattern (3314).

[0335] Additionally, a portion of the lower conductive layer (3370) is connected to the electromagnetic shielding layer (3350) and can function as an electrical path to ground the electromagnetic waves incident on the electromagnetic shielding layer (3350).

[0336] For example, the lower conductive layer (3370) may include a conductive material.

[0337] A thermal conductive film (3380) may be provided on the surface of the first insulating pattern (3314) of the first package (3310), the surface of the outer sealing layer (3360), and / or the surface of the electromagnetic shielding layer (3350) between the first package (3310) and the outer sealing layer (3360). The thermal conductive film (3380) may cover the lower conductive layer (3370) and may include an opening that exposes a portion of the lower conductive layer (3370).

[0338] For example, the thermal conductive film (3380) may include an insulating material with excellent thermal conductivity.

[0339] A semiconductor package-on-package (3000d) according to one example of the present disclosure may include a plurality of stacked packages. Here, the first package (3310) includes a neural processing unit (100), and the plurality of vertically stacked packages (e.g., second to fourth packages (3320, 3330, 3340)) may be stacked packages including one or more memory chips. Here, if the neural processing unit (100) generates more heat than the memory, it may be advantageous for the neural processing unit (100) to be positioned above the memory for heat dissipation.

[0340] The semiconductor package-on-package (3000d) may be configured to include at least one of an electromagnetic shielding layer (3350), an outer sealing layer (3360), a lower conductive layer (3370), and a thermal conductive film (3380) corresponding to a plurality of stacked packages (3310, 3320, 3330, 3340).

[0341] In addition, the technical features of different examples of the present disclosure may be combined and modified.

[0342] FIG. 18 is a cross-sectional view showing a package according to the fifth example.

[0343] Referring to FIG. 18, the package (3000e) according to the fifth example may include a first semiconductor chip (3411) and a second semiconductor chip (3413). The semiconductor package (3000e) may be a system-in-package in which the first semiconductor chip (3411) and the second semiconductor chip (3413), etc., are packaged in a fan-out manner.

[0344] In exemplary embodiments, the first semiconductor chip (3411) may be a logic chip. For example, the first semiconductor chip (3411) may be an NPU. Alternatively, the first semiconductor chip (3411) may be a CPU, MPU, GPU, or AP.

[0345] In exemplary embodiments, the second semiconductor chip (3413) may be a memory semiconductor chip.

[0346] In exemplary embodiments, the second semiconductor chip (3413) may have a cavity capable of accommodating the first semiconductor chip (3411). For example, the second semiconductor chip (3413) may be placed at the periphery of the first semiconductor chip (3411).

[0347] A package (3000e) according to the fifth example may include a sealing layer (3420) that molds the first semiconductor chip (3411) and the second semiconductor chip (3413) so that the first semiconductor chip (3411) and the second semiconductor chip (3413) are integrated. The sealing layer (3420) may cover at least a portion of the first semiconductor chip (3411) and at least a portion of the second semiconductor chip (3413). For example, the sealing layer (3420) may cover the sidewall of the first semiconductor chip (3411) and the lower surface of the first semiconductor chip (3411) provided with a pad (3411p) of the first semiconductor chip (3411), and may cover the sidewall of the second semiconductor chip (3413) and the lower surface of the second semiconductor chip (3413) provided with a pad (3413p) of the second semiconductor chip (3413).

[0348] The package (3000e) according to the fifth example may include a redistribution structure (3430, 3440). The redistribution structure (3430, 3440) may include a redistribution pattern (3430) and an insulation pattern (3440).

[0349] The redistribution pattern (3430) can electrically connect the pad (3411p) of the first semiconductor chip (3411) to the external connection terminal (3190) and the pad (3413p) of the second semiconductor chip (3413) to the external connection terminal (3190). The redistribution pattern (3430) may have a multilayer structure. For example, the redistribution pattern (3430) may include a first sub-redistribution pattern (3431) and a second sub-redistribution pattern (3433). The first sub-redistribution pattern (3431) may extend along the surface of the sealing layer (3420), be connected to the pad (3411p) of the first semiconductor chip (3411), and be connected to the pad (3413p) of the second semiconductor chip (3413). Additionally, a portion of the first sub-rewiring pattern (3431) may extend vertically through the sealing layer (3420).

[0350] The insulation pattern (3440) is provided on the lower surface of the sealing layer (3420) and can cover at least a portion of the first sub-rewiring pattern (3431).

[0351] A protective layer (3450) may be formed on the insulating pattern (3440). The protective layer (3450) may cover the second sub-rewiring pattern (3433) while exposing a portion of the second sub-rewiring pattern (3433). An external connection terminal (3190) may be disposed on the portion of the second sub-rewiring pattern (3433) exposed by the protective layer (3450).

[0352] A heat-conducting film (3460) and a heat sink (3470) may be provided on the upper surface of the first semiconductor chip (3411) and the upper surface of the second semiconductor chip (3413). An adhesive layer (3418) for fixing the first semiconductor chip (3411) may be provided between the upper surface of the first semiconductor chip (3411) and the heat-conducting film (3460), and an adhesive layer (3419) for fixing the second semiconductor chip (3413) may be provided between the upper surface of the second semiconductor chip (3413) and the heat-conducting film (3460). In exemplary embodiments, the adhesive layers (3418, 419) may include a material with excellent thermal conductivity so as to improve heat dissipation for the first semiconductor chip (3411) and heat dissipation characteristics for the second semiconductor chip (3413).

[0353] A package (3000e) according to one example of the present disclosure may include a plurality of horizontally spaced semiconductor chips. Herein, the package (3000e) may include a neural processing unit (100) and one or more memory chips. Herein, the neural processing unit (100) may be located in the center of the package (3000e), and the memory may be placed outside the neural processing unit (100). The package (3000e) may be composed of at least a part of a semiconductor package-on-package.

[0354] In addition, the technical features of different examples of the present disclosure may be combined and modified.

[0355] FIG. 19 is a cross-sectional view showing a package-on-package (PoP) according to the 6th example.

[0356] In FIG. 19, the same descriptions as previously explained are omitted or simplified.

[0357] Referring to FIG. 19, the semiconductor package-on-package (3000f) according to the sixth example may include a lower package (3610) and an upper package (3620). Since the lower package (3610) may have a configuration generally identical to the package (3000e) shown in FIG. 18 and the upper package (3620) may have a configuration generally identical to the semiconductor package-on-package (3000d) shown in FIG. 17, a detailed description of the lower package (3610) and the upper package (3620) is omitted.

[0358] As illustrated in FIG. 19, the first sub-rewiring pattern (3431) provided in the lower package (3610) can extend through the sealing layer (3420) and the thermal conductive film (3380) of the lower package (3610) and can be connected to the lower conductive layer (3370). That is, the first semiconductor chip (3411) of the lower package (3610) and the semiconductor chips (3311, 3321, 3331, 3341) of the upper package (3620) can be connected through the rewiring pattern (3430) of the lower package (3610), the lower conductive layer (3370) of the upper package (3620), and the first to fourth rewiring patterns (3313, 3323, 3333, 3343) of the upper package (3620).

[0359] In exemplary embodiments, the first semiconductor chip (3411) of the lower package (3610) is an NPU, and the semiconductor chips (3311, 3321, 3331, 3341) of the upper package (3620) may be at least one memory semiconductor chip configured to transmit and receive electrical signals with the first semiconductor chip (3411) of the lower package (3610).

[0360] In exemplary embodiments of the present disclosure, the lower package (3610) and the upper package (3620) can be electrically connected without inter-package connection terminals that are susceptible to bending, so the reliability of the semiconductor package-on-package (3000f) can be further improved. Additionally, since multiple packages can be stacked without inter-package connection terminals, a thinner semiconductor package can be manufactured.

[0361] Here, the neural processing unit (100) can be located in the center of the package (3000e), and the memory can be placed outside the neural processing unit (100).

[0362] In addition, the technical features of different examples of the present disclosure may be combined and modified.

[0363] A semiconductor package-on-package (3000f) according to one example of the present disclosure may include a plurality of stacked packages. Here, the package (3000f) may include one or a plurality of neural processing units (100) and one or a plurality of memory chips. The bottom package (3610) of the semiconductor package-on-package (3000f) may be a package including a plurality of horizontally spaced semiconductor chips. Here, although a plurality of vertically stacked packages (e.g., second to fourth packages (3320, 3330, 3340)) are illustrated, the package including the memory chip may be a stack of one or a plurality of packages. Here, if the neural processing unit (100) generates more heat than the memory, it may be advantageous for the neural processing unit (100) to be positioned above the memory for heat dissipation. For example, the top semiconductor chip (3341) may be the neural processing unit (100).

[0364] The semiconductor package-on-package (3000f) may be configured to include at least one of an electromagnetic shielding layer (3350), an outer sealing layer (3360), a lower conductive layer (3370), and a thermal conductive film (3380) corresponding to a plurality of stacked packages (3610, 3310, 3320, 3330, 3340).

[0365] In addition, the technical features of different examples of the present disclosure may be combined and modified.

[0366] IV. 2.5D Package

[0367] Unlike the 3D packaging method, which stacks all semiconductor chips together, the 2.5D packaging method refers to a method where logic chips are arranged in parallel and memory chips are stacked.

[0368] FIG. 20 is a schematic perspective view illustrating a semiconductor package (4000) according to a 2.5D package method, and FIG. 21 is a schematic cross-sectional view of the semiconductor package (4000) illustrated in FIG. 20.

[0369] Referring to FIGS. 20 and 21, the semiconductor package (4000) may include a substrate (4010), an interposer (4020) mounted on the substrate (4010), and processing devices (4030) and memory stacks (4040A, 4040B) mounted side-by-side on the interposer (4020).

[0370] A semiconductor package (4000) according to one example of the present disclosure may include at least one memory chip and at least one neural processing unit (100).

[0371] The above substrate (4010) may include a printed circuit board (PCB).

[0372] Alternatively, the substrate (4010) may be a glass substrate. While conventional substrates mainly consist of silicon or organic materials, these substrates have a high coefficient of thermal expansion, which can reduce reliability and cause dielectric loss during high-speed signal transmission. On the other hand, glass substrates have excellent thermal stability and electrical properties.

[0373] The glass substrate may contain silica (SiO2) and boric acid (B2O3). In this case, physical and electrical properties can be optimized by controlling the content of silica (SiO2) and boric acid (B2O3), and specific properties can be enhanced through additional metal oxide doping. Additionally, the glass substrate may contain alumina (Al2O3). Silica (SiO2) may be included in an amount of 60 to 80%, enabling high heat resistance and mechanical strength, and reducing dielectric loss. Boric acid (B2O3) may be included in an amount of 10 to 25%, enabling a reduction in the coefficient of thermal expansion, improvement in processability, and also providing a high dielectric constant and complementing mechanical strength. Alumina (Al2O3) may be included in an amount of 0 to 10%, enhancing chemical durability and improving mechanical strength, as well as increasing the dielectric constant. Metal oxides for controlling properties may include titania (TiO2), strontium oxide (SrO), magnesia (MgO), and calcium oxide (CaO). Titania (TiO2) may be included in an amount of 1 to 5%, which can increase the dielectric constant while simultaneously maintaining mechanical strength. Strontium oxide (SrO) may be included in an amount of 1 to 5%, which improves high-frequency signal characteristics and enables low dielectric loss. Magnesia (MgO) may be included in an amount of 0 to 5%, which can reduce dielectric loss and increase heat resistance. Calcium oxide (CaO) may be included in an amount of 0 to 3%, which can reduce dielectric loss and improve thermal stability.

[0374] A plurality of outer bumps (4090) may be disposed on the lower surface of the substrate (4010), and a plurality of inner bumps (4080) may be disposed on the upper surface.

[0375] For example, the plurality of external bumps (4090) may be electrically connected to an external system board, main board, or module board, etc. The plurality of external bumps (4090) may include solder balls or metallic bumps. The plurality of internal bumps (4080) may electrically connect the substrate (4010) and the interposer (4020). The plurality of internal bumps (4080) may include solder balls or metallic bumps. The interposer (4020) may have a board form on which the processing unit (4030) and the memory stacks (4040A, 4040B) are mounted, and may provide electrical connections between the processing unit (4030) and the memory stacks (4040A, 4040B).

[0376] However, the present disclosure is not limited to memory stacks (4040A, 4040B) and may be configured to include only one of the memory stacks (4040A, 4040B).

[0377] However, the present disclosure is not limited to the external bumps (4090) of the substrate (4010). For example, referring to FIGS. 25 through 27, the substrate (4010) may be implemented in an M.2 form factor, in which case the external bumps (4090) may be replaced with M.2 slots (9100).

[0378] The processing device (4030) may include one of an NPU (100), a CPU (300), a GPU (400), and an MCU (500).

[0379] For example, the processing device (4030) can send clock signals and / or command signals to the memory stacks (4040A, 4040B) and exchange data signals. Additionally, the processing device (4030) can exchange electrical signals with an external processor, power supply, or input / output device, etc., through the interposer (4020) and the substrate (4010). The substrate (4010) may be referred to as a semiconductor package substrate.

[0380] The memory stacks (4040A, 4040B) may include a first memory stack (4040A) disposed on one side of the processing device (4030) and a second memory stack (4040B) disposed on the other side. The memory stacks (4040A, 4040B) may each include first and second base dies (4050A, 4050B) and a plurality of memory dies (4051A-4054A, 4051B-4054B) stacked on the base dies (4050A, 4050B). For example, the first and second memory stacks (4040A, 4040B) are each shown as having four memory dies (4051A-4054A, 4051B-4054B) stacked on them.

[0381] However, although it was described above that there are two memory stacks, this is merely an example, and the number of memory stacks is not limited and can be determined according to the computational capability of the processing device (4030).

[0382] The base dies (4050A, 4050B) may include memory cells, test logic circuits such as DFT (design for test), JTAG (Joint Test Action Group), MBIST (memory built-in self-test), signal interface circuits such as PHY, etc. At least one of the base dies (4050A, 4050B) may be configured to include a plurality of memory cells.

[0383] The plurality of memory dies (4051A-4054A, 4051B-4054B) may include the memory (200) illustrated in FIG. 6. The plurality of memory dies (4051A-4054A, 4051B-4054B) may be volatile memory (e.g., SRAM, DRAM, or LPDDR). The plurality of memory dies (4051A-4054A, 4051B-4054B) may be non-volatile memory (e.g., MRAM (Magnetic RAM), ReRAM (Resistive RAM). The memory (200) may include at least one or a plurality of memories (200-1, 200-2).

[0384] The type and number of memory dies may be determined based on the processing performance of the NPU (100) (e.g., the NPU's Tera Operations Per Second (TOPS)). For example, a memory bandwidth of 2 GB / s to 4 GB / s per 1 TOPS may be required. For example, 10 TOPS of NPU may have memory configured to provide a memory bandwidth of 20 GB / s to 40 GB / s. The memory bandwidth may be determined based on the memory's operating frequency and the number of communication channels.

[0385] For example, to configure a memory bandwidth of 40 GB / s with LPDDR5, the following formula can be used: Bandwidth (GB / s) = Data Rate (Gbps per pin) × Bus Width (bits) × Number of Channels χ 8. Data Rate: LPDDR5 generally supports a maximum data rate of 6400 Mbps per pin. Bus Width: The bus width of each LPDDR channel is typically 16 bits (2 bytes). Note that the number of channels cannot be a decimal and must be an integer.

[0386] Each of the above-mentioned multiple memory dies (4051A-4054A, 4051B-4054B) may include TSVs (through silicon vias) (4060A, 4060B). Each of the above-mentioned TSVs (4060A, 4060B) may vertically penetrate the above-mentioned multiple memory dies (4051A-4054A, 4051B-4054B). Each of the above-mentioned TSVs (4060A, 4060B) may include central TSVs (4061A, 4061B) and non-central TSVs (4062A, 4062B). The central TSVs (4061A, 4061B) may penetrate the central region of the above-mentioned memory stacks (4040A, 4040B). The non-central TSVs (4062A, 4062B) may penetrate the non-central region of the memory stacks (4040A, 4040B). For example, the central region may be an area adjacent to a virtual center vertical line dividing the memory stacks (4040A, 4040B) in two, and the non-central region may be an area spaced apart from the virtual vertical line. The distances from the processing device (4030) to the central TSVs (4061A, 4061B) may each be greater than the distances from the processing device (4030) to the non-central TSVs (4062A, 4062B). Alternatively, electrical paths from the processing device (4030) to the non-central TSVs (4062A, 4062B) may each be smaller than electrical paths from the processing device (4030) to the central TSVs (4061A, 4061B). The non-central TSVs (4062A, 4062B) may include first non-central TSVs (4062Aa, 4062Ba) positioned relatively close to the outer side of the memory stacks (4040A, 4040B), and second non-central TSVs (4062Ab, 4062Bb) positioned relatively close to the central region (40 or a virtual center vertical line) of the memory stacks (4040A, 4040B).Specifically, the first non-central TSVs (4062Aa, 4062Ba) may be located closest to the processing device (4030), the second non-central TSVs (4062Ab, 4062Bb) may be located closest to the processing device (4030), and the central TSVs (4061A, 4061B) may be located farthest from the processing device (4030). The distances of the first non-central TSVs (4062Aa, 4062Ba) from the processing device (4030) may each be smaller than the distances from the processing device (4030) to the second non-central TSVs (4062Ab, 4062Bb). The electrical paths passing through the central TSVs (4060A, 4061B) adjacent to the virtual center vertical line from the processing device (4030) may each be longer than the electrical paths passing through the TSVs (4062A, 4062B) spaced apart from the virtual vertical line from the processing device (4030).

[0387] Processor micro-bumps (4035A, 4035B, 4035C) may be disposed between the interposer (4020) and the processing device (4030). The processor micro-bumps (4035A, 4035B, 4035C) may include first processor micro-bumps (4035A) for electrically connecting to the first base die (4050A) of the first memory stack (4040A), second processor micro-bumps (4035B) for electrically connecting to the second base die (4050B) of the second memory stack (4040B), and central processor micro-bumps (4035C) for electrically connecting to an external electronic device through the interposer (4030) and the substrate (4010). The processor micro-bumps (4035A, 4035B, 4035C) may include solder balls or metallic bumps. The first processor micro-bumps (4035A) may be positioned close to the first memory stack (4040A), and the second processor micro-bumps (4035B) may be positioned close to the second memory stack (4040B).

[0388] First memory micro-bumps (4045A, 4046A) may be disposed between the interposer (4020) and the first base die (4050A) of the first memory stack (4040A), and second memory micro-bumps (4045B, 4046B) may be disposed between the interposer (4020) and the second base die (4050B) of the second memory stack (4040B). The first memory micro-bumps (4045A, 4046A) may include first outward memory micro-bumps (4045A) and first inward memory micro-bumps (4046A), and the second memory micro-bumps (4045B, 4046B) may include second outward memory micro-bumps (4045B) and second inward memory micro-bumps (4046B). The inward and outward references may be determined based on the processing device (4030). The first and second outward memory micro-bumps (4045A, 45B) may be electrically connected to an external electronic device through the interposer (4020), the inner bumps (4080), the substrate (4010), and the outer bumps (4090). However, as described above, the external bumps (4090) may be modified and replaced with sockets, connectors, etc. The first inward memory micro-bumps (4046A) may be electrically connected to the first processor micro-bumps (4035A) through the first internal connection (4047A) inside the interposer (4020), and the second inward memory micro-bumps (4046B) may be electrically connected to the second processor micro-bumps (4035B) through the second internal connection (4047B) inside the interposer (4020).The first and second inward memory micro-bumps (4046A, 4046B) may each be positioned closer to the processing device (4030) than the first and second outward memory micro-bumps (4045A, 4045B). That is, the distances from the processing device (4030) to the first and second outward memory micro-bumps (4045A, 4045B) may each be greater than the distances from the processing device (4030) to the first and second inward memory micro-bumps (4046A, 4046B).

[0389] Inter-die micro-bumps (4049A, 49B) may be disposed between the memory dies (4051A-4054A, 4051B-4054B), that is, between the TSVs (4060A, 4060B), respectively. The inter-die micro-bumps (4049A, 49B) may electrically connect the TSVs (4060A, 4060B). The inter-die micro-bumps (4049A, 49B) may include solder balls or metallic bumps.

[0390] The semiconductor package (4000) according to an embodiment of the present disclosure may have relatively short electrical paths through TSVs (4062A, 4062B) spaced apart from the virtual vertical line and relatively long electrical paths through TSVs (4061A, 4061B) adjacent to the virtual vertical line, extending into the memory elements (4051A-4054A, 4051B-4054B) of the processing device (4030) and the memory stacks (4040A, 4040B).

[0391] FIG. 22 is a schematic top view showing the top surface of the interposer (4020) illustrated in FIG. 20.

[0392] For example, a ball grid array of the interposer (4020) is schematically illustrated. Referring to FIG. 22, the interposer (4020) may include a processing device mounting area (4023) and first and second memory stack mounting areas (4024A, 4024B). Here, the first and second memory stack mounting areas (4024A, 4024B) may be referred to as at least one memory mounting area.

[0393] The processing device mounting area (4023) may include a first processing device mounting area (4023A), a second processing device mounting area (4023B), and a central processing device mounting area (4023C). The first and second processing device mounting areas (4023A, 4023B) may each provide an area where pads, lands, or connectors can be formed for the placement of the first and second processor micro-bumps (4035A, 4035B) to be electrically connected to the first and second memory stacks (4040A, 4040B). A land is a term primarily used in semiconductor packages or printed circuit boards (PCBs) and refers to a solder pad for electrical connection. Simply put, it is a metal surface designed to connect electrical signals or power. The first processing device mounting area (4023A) may be positioned close to the first memory stack mounting area (4024A), and the second processing device mounting area (4023B) may be positioned close to the second memory stack mounting area (4024B). The central processing device mounting area (4023C) may provide an area where pads, lands, or connectors may be formed for the central processor micro-bumps (4035C) to be placed. The central processing device mounting area (4023C) may be positioned in the central area of ​​the processing device mounting area (4023), for example, between the first processing device mounting area (4023A) and the second processing device mounting area (4023B).

[0394] The first and second memory stack mounting regions (4024A, 4024B) may each include first and second outward memory stack mounting regions (4025A, 4025B) on which pads, lands, or connectors may be formed for the placement of the first and second outward memory micro-bumps (4045A, 4046A). Additionally, the first and second memory stack mounting regions (4024A, 4024B) may each include first and second inward memory stack mounting regions (4026A, 4026B) on which pads, lands, or connectors may be formed for the placement of the first and second inward memory micro-bumps (4046A, 4046B). The first and second inward memory stack mounting regions (4026A, 4026B) may be positioned close to the processing device mounting region (4023). That is, the distances from the processing device mounting region (4023) to the first and second outward memory stack mounting regions (4025A, 4025B) may each be greater than the distances from the processing device mounting region (4023) to the first and second inward memory stack mounting regions (4026A, 4026B). In the drawing, the micro-bumps (4035A, 4035B, 4035C, 4045A, 4045B, 4046A, 4046B) may be pads, lands, or connectors.

[0395] FIG. 23 is a schematic bottom view of the first and second base dies (4050A, 4050B) of the first and second memory stacks (4040A, 4040B) shown in FIG. 20.

[0396] For example, a ball grid array of the lower surfaces of the first and second base dies (4050A, 4050B) is illustrated. Referring to FIG. 23, the lower surfaces of the first and second base dies (4050A, 4050B) may each include first and second outward memory ball grid arrays (4055A, 4055B) on which pads, lands, or connectors may be formed for the placement of the first and second outward memory micro-bumps (4045A, 4045B). Additionally, the lower surfaces of the first and second base dies (4050A, 4050B) may each include first and second inward memory ball grid arrays (4056A, 4056B) on which pads, lands, or connectors may be formed for the placement of the first and second inward memory micro-bumps (4046A, 4046B). The first and second inward memory ball grid arrays (4056A, 4056B) may be placed closer to the processing device (4030) than the first and second outward memory ball grid arrays (4055A, 4055B). The term "inward" means a direction closer to the processing device (4030), and the term "outward" means a direction further away from the processing device. That is, the distances from the processing device (4030) to the first and second outward memory ball grid arrays (4055A, 4055B) may each be greater than the distances from the processing device (4030) to the first and second inward memory ball grid arrays (4056A, 4056B). In the drawing, the micro-bumps (4045A, 4045B, 4046A, 4046B) may be pads, lands, or connectors.

[0397] FIG. 24 is a schematic top view of the first and second base dies (4050A, 4050B) of the first and second memory stacks (4040A, 4040B) shown in FIG. 20.

[0398] For example, a ball grid array of the top surfaces of the first and second base dies (4050A, 4050B) is illustrated. Referring to FIG. 24, the top surfaces of the first and second base dies (4050A, 4050B) may each include circuit occupied regions (4065A, 4065B), central TSV occupied regions (4066A, 4066B), PHY occupied regions (4067Aa-4067Ac, 4067Ba-4067Bc), and non-central TSV occupied regions (4068Aa-4068Ab, 4068Ba-4068Bb).

[0399] The circuit occupancy regions (4065A, 4065B) may be regions in which test logic circuits such as the DFT circuit, the JTAG circuit, and the MBIST circuit can be placed. Additionally, the circuit occupancy regions (4065A, 4065B) may be modified to include a plurality of memory cells. The circuit occupancy regions (4065A, 4065B) may be positioned adjacent to the outer sides of the first and second base dies (4050A, 4050B) so as to be furthest from the processing device (4030).

[0400] The central TSV occupied regions (4066A, 4066B) may each be positioned adjacent to imaginary centerlines that bisect the upper surfaces of the first and second base dies (4050A, 4050B) in a column direction. In detail, the central TSV occupied regions (4066A, 4066B) may each be positioned between the circuit occupied regions (4065A, 4065B) and imaginary vertical centerlines that bisect the upper surfaces of the first and second base dies (4050A, 4050B). With further reference to FIG. 21, the central TSV occupied regions (4066A, 4066B) may be regions that the central TSVs (4061A, 4061B) can occupy. That is, the central TSV occupied regions (4066A, 66B) can each be vertically overlapped with the central TSVs (4061A, 4061B). The distances from the processing device (4030) to the circuit occupied regions (4065A, 4065B) can each be greater than the distances from the processing device (4030) to the central TSV occupied regions (4066A, 4066B).

[0401] With further reference to FIG. 21, the above non-central TSV occupied regions (4068Aa-4068Ab, 4068Ba-4068Bb) may be regions that the non-central TSVs (4062A, 4062B) can occupy. That is, the above non-central TSV occupied regions (4068Aa-4068Ab, 4068Ba-4068Bb) may overlap vertically with the above non-central TSVs (4062A, 4062B). This will be explained with further reference to FIG. 21. The above non-central TSV occupied regions (4068Aa-4068Ab, 4068Ba-4068Bb) may include first non-central TSV occupied regions (4068Aa, 4068Ba) that are relatively closer to the sides of the base dies (4050A, 4050B) to overlap with the first non-central TSVs (4062Aa, 4062Ba), and second non-central TSV occupied regions (4068Ab, 4068Bb) that are relatively closer to the virtual centerlines of the memory stacks (4040A, 4040B) to overlap with the second non-central TSVs (4062Ab, 4062Bb).

[0402] The above pie occupancy regions (4067Aa-4067Ac, 4067Ba-4067Bc) are first pie occupancy regions (4067Aa, 4067Ba) positioned between the sides of the base dies (4050A, 4050B) and the first non-central TSV occupancy regions (4068Aa, 4068Ba) so as to be close to the processing device (4030), second pie occupancy regions (4067Ab, 4067Bb) positioned between the central TSV occupancy regions (4066A, 4066B) and the second non-central TSV occupancy regions (4068Ab, 4068Bb), and the first non-central TSV occupancy regions (4068Aa, 4068Ba) and the second non-central TSV occupancy It may include third pi-occupied regions (4067Ac, 4067Bc) positioned between regions (4068Ab, 4068Bb). The distances from the processing device (4030) to the first non-central TSV occupied regions (4068Aa, 4068Ba) may each be smaller than the distances from the processing device (4030) to the second non-central TSV occupied regions (4068Ab, 4068Bb). The distances from the processing device (4030) to the second non-central TSV occupied regions (4068Ab, 4068Bb) may each be larger than the distances from the processing device (4030) to the third non-central TSV occupied regions (4068Ac, 4068Bc). Referring further to FIG. 21, the pi-occupied regions (4067Aa-4067Ac, 4067Ba-4067Bc) may be regions that overlap with the inward memory micro-bumps (4046A, 4046B).

[0403] The inter-die micro-bumps (4049A, 49B) may be disposed within the central TSV occupied regions (4066A, 4066B). The inter-die micro-bumps (4049A, 4049B) may electrically connect the central TSVs (4061A, 4061B) and electrically connect the non-central TSVs (4062A, 4062B) within the non-central TSV occupied regions (4068Aa-4068Ab, 4068Ba-4068Bb). In the drawing, the inter-die micro-bumps (4049A, 4049B) may be pads, lands, or connectors.

[0404] The NPU (100) illustrated in FIG. 3 can be mounted on a small electronic device. For example, the NPU (100) can be mounted on a CCTV (closed-circuit television) camera, etc. To be mounted on the CCTV camera, etc., a form factor having an M.2 slot can be used.

[0405] Figure 25 shows an example of a form factor with an M.2 slot.

[0406] The M.2 slot is a next-generation interface standard for high-speed data transfer that offers high speed and efficiency. Designed with a compact size, it is being adopted in various devices such as laptops, desktops, and mini PCs. In particular, small form factors like M.2 make it easy to apply on-device.

[0407] The form factor (9000) illustrated in FIG. 25 has a width of 22 mm. The length may vary, such as 30 mm, 42 mm, 60 mm, 80 mm, 110 mm, etc., but can be selected according to the physical constraints of the device to be mounted. The size of the package (5000) in which the NPU and memory are mounted together is constrained by the width of the form factor (9000) of 22 mm and the minimum length of 30 mm. That is, the maximum width of the package (5000) must be smaller than the width of the form factor (9000) of 22 mm.

[0408] Therefore, below, methods for miniaturizing the package (5000) in which the NPU and memory are mounted will be explained.

[0409] In some examples, the form factor may be E1.S. The E1.S interface is a form factor designed as a standardized high-density solution for data center and enterprise storage systems. E1.S supports various protocols and applications to provide high-speed data transfer rates and efficient space utilization. E1.S modules are rectangular and typically have a width of 31.5 mm to 38.4 mm and a length of 111.49 mm to 318.75 mm. E1.S can support the Peripheral Component Interconnect Express (PCIe) interface. In the case of the PCIe interface, much higher data transfer speeds can be provided by utilizing multiple lanes (x2, x4, x8, etc.).

[0410] The above package (5000) may be configured to include at least one semiconductor chip to which a fan-in semiconductor package, a fan-out semiconductor package, a package-on-package (PoP), and / or a 2.5D package method is applied.

[0411] In addition, the technical features of different examples of the present disclosure may be combined and modified.

[0412] FIG. 26 is an exemplary diagram showing a package (5000) according to the first method of the present disclosure.

[0413] Referring to FIG. 26, the length of the form factor (9000) with an M.2 slot may be 42 mm, 60 mm, 80 mm, or 110 mm.

[0414] A package (5000) mounted on a form factor (9000) may include an NPU (100) and a plurality of memories (200-1, 200-2, 200-3, 200-4).

[0415] The above NPU (100) may include a plurality of processing elements (PE) (110), SFU (150), NPU internal memory (120), NPU controller (130), and NPU interface (140), as shown in FIG. 3 or FIG. 5.

[0416] The above multiple memories (200-1, 200-2, 200-3, 200-4) may be Static Random Access Memory (SRAM). Unlike Dynamic Random Access Memory (DRAM), which requires periodic updates to its contents, SRAM retains its contents as long as power is supplied to the memory device. Due to the symmetrical structure of the circuit, SRAM enables much faster input / output than DRAM. Additionally, unlike DRAM, which requires two accesses in the order of the upper bit and the lower bit when accessing a memory address, SRAM has the advantage of being able to access it in a single step. Consequently, SRAM is faster and has a wider bandwidth than DRAM.

[0417] Meanwhile, in SRAM, each bit is stored in two pairs of inverters, each consisting of four transistors. The two pairs of inverters maintain the values ​​of 0 and 1 in a stable state, while two access transistors perform read and write functions. Therefore, generally six transistors are required to store one bit.

[0418] As a result, a disadvantage arises in that the physical size of the SRAM increases while the storable data size decreases. In other words, because SRAM has a lower integration density than DRAM, the number of bits that can be stored relative to the area is small. The physical size of the NPU (100) may be, for example, 4.50 mm x 4.85 mm. Considering the minimum separation distance required between the NPU (100) and the plurality of memories (200-1, 200-2, 200-3, 200-4), the physical size of each memory may be limited to 6.43 mm x 5.30 mm. As mentioned above, because SRAM has a low integration density, if the physical size decreases, the number of bits that can be stored also decreases.

[0419] Therefore, the semiconductor manufacturing process of SRAM can be lower than that of NPU. For example, the semiconductor manufacturing process of SRAM may utilize 14nm, while the semiconductor manufacturing process of NPU may utilize 7nm. For example, the semiconductor manufacturing process of SRAM may utilize 5nm, while the semiconductor manufacturing process of NPU may utilize 7nm. For example, the semiconductor manufacturing process of SRAM may utilize 2nm, while the semiconductor manufacturing process of NPU may utilize 5nm.

[0420] However, if the performance of the NPU is 10 Tera Operations Per Second (TOPS), the SRAM must provide a memory bandwidth of 20 GB / s to 40 GB / s, so the SRAM must be arranged in multiple numbers. For this reason, in FIG. 26, the number of memory, i.e., SRAM, is shown as 4.

[0421] And, if the NPU (100) and the plurality of memories (200-1, 200-2, 200-3, 200-4) are placed closest to each other, the size of the package can be 21mm x 31mm.

[0422] Accordingly, the above package (5000) can be mounted on a form factor (9000) having a width of 22 mm and a length of 42 mm.

[0423] However, the package (5000) illustrated in FIG. 26 cannot be mounted on a form factor (9000) having a length of 30 mm. Therefore, below, with reference to FIG. 27, a method for mounting a package (5000) including an NPU (100) and the plurality of memories (200-1, 200-2, 200-3, 200-4) on a form factor (9000) having a length of 30 mm will be described.

[0424] The above package (5000) may be configured to include at least one semiconductor chip to which a fan-in semiconductor package, a fan-out semiconductor package, a package-on-package (PoP), and / or a 2.5D package method is applied.

[0425] In addition, the technical features of different examples of the present disclosure may be combined and modified.

[0426] FIG. 27 is an exemplary diagram showing a package (5000) according to the second method of the present disclosure.

[0427] The NPU (100) illustrated in FIG. 27 may include a plurality of processing elements (PE) (110), SFU (150), NPU internal memory (120), NPU controller (130), and NPU interface (140), as illustrated in FIG. 3 or FIG. 5.

[0428] The above plurality of memories (200-1, 200-2, 200-3, 200-4) may be SRAM.

[0429] The NPU (100) and the SRAM may be manufactured using different semiconductor process technologies. For example, the NPU (100) may be manufactured using 5nm semiconductor process technology, and the SRAM may be manufactured using 2nm semiconductor process technology. That is, the SRAM and the NPU (100) may each be manufactured using different semiconductor manufacturing processes.

[0430] The width of the form factor (9000) shown in Fig. 27 is 22 mm and the length is 30 mm.

[0431] In order to mount the package (5000) within such a limited size, the arrangement of the NPU (100) and the plurality of memories (200-1, 200-2, 200-3, 200-4) within the package (5000) must be improved.

[0432] The physical size of the above NPU (100) may be 4.50 mm x 4.85 mm, and the physical size of each memory may be 6.43 mm x 5.30 mm.

[0433] As can be seen by referring to FIG. 27, the NPU (100) and the plurality of memories (200-1, 200-2, 200-3, 200-4) may be arranged parallel to the diagonal of the package (5000).

[0434] By doing this, the size of the above package can be reduced to 21mm x 21mm.

[0435] Referring to FIG. 27, the configuration of multiple semiconductor chips arranged inside a package (5000) can be referred to as a "rotated multi-chip substrate layout." The package (5000) of the rotated multi-chip substrate layout has the effect of reducing the size of the package (5000) due to the arrangement of each semiconductor chip rotated at a specific angle.

[0436] The type and number of memory chips can be determined based on the processing performance of the NPU (100) (e.g., the NPU's Tera Operations Per Second (TOPS)). For example, a memory bandwidth of 2 GB / s to 4 GB / s per TOPS may be required. For example, if the NPU's performance is 10 Tera Operations Per Second (TOPS), the SRAM must provide a memory bandwidth of 20 GB / s to 40 GB / s, so multiple SRAMs must be arranged. For this reason, the number of memory, i.e., SRAM, is shown as 4 in FIG. 27.

[0437] Meanwhile, the number of memory chips may be changed by taking into account the processing capability of the neural processing unit (100) (e.g., TOPS), the size of a specific form factor, and the size that can be placed within a package smaller than the specific form factor.

[0438] FIG. 28 is an example diagram showing a cross-sectional view along the line A-A' of the package (5000) illustrated in FIG. 26 or FIG. 27.

[0439] Referring to FIG. 28, an NPU (100) and a plurality of memories (only 200-2 and 200-4 are shown in FIG. 28) can be mounted on a substrate (5950).

[0440] The substrate (5950) may be an interposer. That is, the package (5000) may be in the form of a fan-out semiconductor package. Solder bumps may be formed on the lower surface of the substrate (5950). The substrate (5950) may be referred to as a semiconductor package substrate. Alternatively, the substrate (5950) may be a glass substrate.

[0441] The glass substrate may include silica (SiO2) and boric acid (B2O3). In this case, physical and electrical properties can be optimized by controlling the content of silica (SiO2) and boric acid (B2O3), and specific properties can be enhanced through additional metal oxide doping. Additionally, the glass substrate may include alumina (Al2O3).

[0442] An outer sealing layer (5920) is covered over the upper part of the package (5000) to protect the NPU (100) and the plurality of memories. The outer sealing layer (5920) may include a material with excellent thermal conductivity to improve the heat dissipation characteristics of the package (5000).

[0443] A thermal conductive film (5910) may be attached between the outer sealing layer (5920) and the NPU (100). The thermal conductive film (3380) may include an insulating material with excellent thermal conductivity.

[0444] The aforementioned multiple memories may be SRAMs. Unlike DRAM (Dynamic Random Access Memory), which requires periodic updates, SRAM retains its contents as long as power is supplied to the memory device. Due to the symmetrical structure of its circuitry, SRAM enables much faster input / output than DRAM. Furthermore, unlike DRAM, which requires two accesses—first the upper bit and then the lower bit—when accessing a memory address, SRAM has the advantage of allowing a single access. Consequently, SRAM is faster and has a wider bandwidth than DRAM.

[0445] Meanwhile, in SRAM, each bit is stored in two pairs of inverters, each consisting of four transistors. The two pairs of inverters maintain the values ​​of 0 and 1 in a stable state, while two access transistors perform read and write functions. Therefore, generally six transistors are required to store one bit.

[0446] As a result, a disadvantage arises where the physical size of SRAM increases while the storable data size decreases. In other words, because SRAM has a lower integration density compared to DRAM, it can store fewer bits per unit area.

[0447] The physical size of the NPU (100) may be, for example, 4.50 mm x 4.85 mm. Considering the minimum separation distance required between the NPU (100) and the plurality of memories (200-1, 200-2, 200-3, 200-4), the physical size of each memory may be limited to 6.43 mm x 5.30 mm. As previously mentioned, since SRAM has low density, if the physical size is reduced, the number of bits that can be stored also decreases.

[0448] If the performance of the NPU is 10 Tera Operations Per Second (TOPS), the SRAM must provide a memory bandwidth of 20 GB / s to 40 GB / s, so the SRAM must be arranged in multiple quantities.

[0449] <Summary of Examples>

[0450] According to one example of the present disclosure, an NPU semiconductor package is provided. The NPU semiconductor package may include: a semiconductor package substrate; an NPU chip mounted on the semiconductor package substrate, rotated at a first angle relative to one side of the semiconductor package substrate with respect to a plane; and one or more Static Random Access Memory (SRAM) chips mounted on the semiconductor package substrate adjacent to the NPU chip, rotated at a second angle relative to one side of the semiconductor package substrate with respect to a plane. The first angle and the second angle may be determined such that the size of the semiconductor package substrate is smaller than the size of a specific form factor.

[0451] The first angle and the second angle may be the same as each other.

[0452] The first angle and the second angle may be different from each other.

[0453] The above NPU chip and the above SRAM chip may be manufactured using different semiconductor manufacturing processes.

[0454] The first angle and the second angle may correspond to opposite diagonals of the semiconductor package substrate.

[0455] The size of the semiconductor package substrate may be 21mm x 31mm or less.

[0456] The number of the above plurality of SRAM chips may be 2 to 4.

[0457] The plurality of SRAM chips may include: a first SRAM chip disposed in a first direction of the NPU, a second SRAM chip disposed in a second direction of the NPU, a third SRAM chip disposed in a third direction of the NPU, and a fourth SRAM chip disposed in a fourth direction of the NPU.

[0458] The above NPU semiconductor package can be mounted in an M.2 form factor.

[0459] The above M.2 form factor may have a width of 22 mm and a length of 30 mm to 110 mm.

[0460] The above NPU semiconductor package can be mounted in an E1.S form factor.

[0461] The above E1.S form factor may have a width of 31.5 mm to 38.4 mm and a length of 111.49 mm to 318.75 mm.

[0462] According to one example of the present disclosure, an NPU semiconductor package is also provided. The NPU semiconductor package may include: a substrate; an NPU mounted on the substrate, rotated at a first angle relative to one side of the substrate with respect to a plane; and one or more SRAMs mounted on the substrate adjacent to the NPU, rotated at a second angle relative to one side of the substrate with respect to a plane. The first angle and the second angle may be adjusted so that the size of the substrate is smaller than the size of a specific form factor.

[0463] The above NPU may further include an NPU core comprising a plurality of processing elements (PEs), NPU internal memory, an NPU controller, and an NPU interface.

[0464] The NPU is positioned in the center of the substrate, and the plurality of SRAMs can be positioned on the outer side of the substrate.

[0465] The above NPU is a system-on-chip, and the system-on-chip may further include at least one of an ISP (Image Signal Processor), CPU (Central Processor Unit), MPU (Micro Processor Unit), GPU (Graphics Processing Unit), or AP (Application Processor) electrically connected to the NPU core.

[0466] The above NPU semiconductor package may include at least one of an electromagnetic shielding layer, an outer sealing layer, a lower conductive layer, and a thermal conductive film.

[0467] According to one example of the present disclosure, an AI processor package is provided. The AI ​​processor package may include an AI processor specialized for AI computation; and one or more SRAMs electrically connected to the AI ​​processor. The AI ​​processor and the one or more SRAMs may be arranged in a rotated multi-chip substrate layout to form a rotated multi-chip substrate package.

[0468] The size of the AI ​​processor package may be smaller than the size of the rotated multi-chip substrate layout above compared to the non-rotated multi-chip substrate layout.

[0469] The AI ​​processor or the one or more SRAMs may be implemented as one of a flip-chip package, a WLCSP (Wafer Level Chip Scale Package) package, or a TSV (Through Silicon Via) package.

[0470] The above one or more SRAMs can be implemented as a semiconductor package-on-package.

[0471] According to another example of the present disclosure, a Neural Processing Unit (NPU) semiconductor package is provided. The NPU semiconductor package may include a semiconductor package substrate having solder bumps formed on its lower surface; an NPU mounted on the semiconductor package substrate in a diagonal direction opposite to the semiconductor package substrate; and a plurality of SRAMs mounted on the semiconductor package substrate to surround the NPU in the diagonal direction.

[0472] The size of the semiconductor package substrate may be 21mm x 31mm or 21mm x 21mm.

[0473] The above plurality of SRAMs may include four SRAMs.

[0474] The plurality of SRAMs may include: a first SRAM arranged parallel to the diagonal of the semiconductor package substrate in the upward direction of the NPU; a second SRAM arranged parallel to the diagonal of the semiconductor package substrate in the left direction of the NPU; a third SRAM arranged parallel to the diagonal of the semiconductor package substrate in the right direction of the NPU; and a fourth SRAM arranged parallel to the diagonal of the semiconductor package substrate in the downward direction of the NPU.

[0475] The above NPU semiconductor package can be mounted in an M.2 form factor.

[0476] The above M.2 form factor has a width of 22 mm, and a length of 30 mm, 42 mm, 60 mm, 80 mm, and 110 mm.

[0477] A thermal conductive film may be attached to the upper surface of the above NPU.

[0478] An external sealing layer (5920) is covered on the upper surface of the above package to protect the NPU and the plurality of SRAMs.

[0479] The above semiconductor package substrate may include an interposer substrate.

[0480] The physical size of the above NPU may be 4.50 mm x 4.85 mm.

[0481] The physical size of each of the above plurality of SRAMs may be 6.43mm x 5.30mm.

[0482] The above NPU may include: a plurality of processing elements (PEs), NPU internal memory, an NPU controller, and an NPU interface.

[0483] Although examples of the present disclosure have been described in more detail with reference to the attached drawings, the present disclosure is not necessarily limited to these examples and may be modified in various ways within the scope of the technical concept of the present disclosure. Accordingly, the examples disclosed in the present disclosure are intended to explain, not limit, the technical concept of the present disclosure, and the scope of the technical concept of the present disclosure is not limited by these examples. Therefore, the examples described above should be understood as illustrative in all respects and not restrictive. The scope of protection of the present disclosure shall be interpreted by the claims below, and all technical concepts within an equivalent scope shall be interpreted as being included within the scope of rights of the present disclosure.

Claims

Claim 1 A semiconductor package substrate; an NPU chip mounted on the semiconductor package substrate, rotated at a first angle relative to one side of the semiconductor package substrate with respect to a plane; and one or more SRAM (Static Random Access Memory) chips mounted on the semiconductor package substrate adjacent to the NPU chip, rotated at a second angle relative to one side of the semiconductor package substrate with respect to a plane; wherein the first angle and the second angle are determined such that the size of the semiconductor package substrate is smaller than the size of a specific form factor. Claim 2 In claim 1, the first angle and the second angle are the same or different from each other in an NPU semiconductor package. Claim 3 In claim 1, the NPU semiconductor package, wherein the NPU chip and the SRAM chip are manufactured using different semiconductor manufacturing processes. Claim 4 In claim 1, the first angle and the second angle correspond to opposite diagonals of the semiconductor package substrate, an NPU semiconductor package. Claim 5 An NPU semiconductor package according to claim 1, wherein the size of the semiconductor package substrate is 21mm x 31mm or less. Claim 6 An NPU semiconductor package according to claim 1, wherein the number of the plurality of SRAM chips is 2 to 4. Claim 7 In claim 1, the plurality of SRAM chips comprises: a first SRAM chip disposed in a first direction of the NPU, a second SRAM chip disposed in a second direction of the NPU, a third SRAM chip disposed in a third direction of the NPU, and a fourth SRAM chip disposed in a fourth direction of the NPU, wherein the NPU semiconductor package comprises Claim 8 In claim 1, the NPU semiconductor package is an NPU semiconductor package mounted in an M.2 form factor. Claim 9 In claim 8, the M.2 form factor is an NPU semiconductor package having a width of 22 mm and a length of 30 mm to 110 mm. Claim 10 In claim 1, the NPU semiconductor package is an NPU semiconductor package mounted in an E1.S form factor. Claim 11 In claim 10, the above E1.S form factor is an NPU semiconductor package having a width of 31.5 mm to 38.4 mm and a length of 111.49 mm to 318.75 mm. Claim 12 A semiconductor NPU package comprising: a substrate; an NPU mounted on the substrate, rotated at a first angle relative to one side of the substrate relative to a plane; and one or more SRAMs mounted on the substrate adjacent to the NPU, rotated at a second angle relative to one side of the substrate relative to a plane, wherein the first angle and the second angle are adjusted such that the size of the substrate is smaller than the size of a specific form factor. Claim 13 In claim 12, the NPU further comprises: an NPU semiconductor package including a plurality of processing elements (PEs), an NPU internal memory, an NPU controller, and an NPU interface, wherein the NPU further comprises an NPU core. Claim 14 In claim 12, the NPU is positioned in the center of the substrate, and the plurality of SRAMs are positioned on the outer side of the substrate, in an NPU semiconductor package. Claim 15 In claim 12, the NPU is a system-on-chip, and the system-on-chip further comprises at least one of an Image Signal Processor (ISP), a Central Processor Unit (CPU), a Micro Processor Unit (MPU), a Graphics Processing Unit (GPU), or an Application Processor (AP) electrically connected to the NPU core, in an NPU semiconductor package. Claim 16 In claim 12, the NPU semiconductor package comprises at least one of an electromagnetic shielding layer, an outer sealing layer, a lower conductive layer, and a thermal conductive film. Claim 17 An AI processor package comprising: an AI processor specialized for AI computation; and one or more SRAMs electrically connected to the AI ​​processor; wherein the AI ​​processor and the one or more SRAMs are arranged in a rotated multi-chip substrate layout to form a rotated multi-chip substrate package. Claim 18 In claim 17, the AI ​​processor package, wherein the rotated multi-chip substrate layout has a smaller size than the non-rotated multi-chip substrate layout. Claim 19 In claim 17, the AI ​​processor or the one or more SRAMs are implemented in one of a flip-chip package, a WLCSP (Wafer Level Chip Scale Package) package, or a TSV (Through Silicon Via) package, an AI processor package. Claim 20 In claim 17, the above one or more SRAMs are implemented as a semiconductor package-on-package, an AI processor package.

Images