System and method capable of detecting system component failure

Abstract

A system is proposed, which is capable of testing components in the system during run time. The system may include: a substrate; a plurality of functional components, the plurality of functional components being mounted on the substrate and including circuits; a system bus formed on the substrate using a conductive pattern so as to allow the plurality of functional components to communicate with each other; one or more wrappers, each of the one or more wrappers being connected to one of the plurality of functional components; and an in-system component tester (ICT), which is configured to: select at least one functional component in an idle state among the plurality of functional components as a component under test (CUT); and test the at least one functional component selected as the CUT through the one or more test wrappers.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to Korean Patent Application No. 10-2020-0189414 filed on December 31, 2020, with the Korean Intellectual Property Office, and Korean Patent Application No. 10-2021-0181082 filed on December 16, 2021, with the Korean Intellectual Property Office, the disclosures of which are incorporated herein by reference. Technical Field

[0003] The present disclosure relates to the detection of failures in a system at runtime. Background Art

[0004] Artificial intelligence (AI) is still developing gradually. AI refers to artificial intelligence that imitates human intelligence, that is, intelligence used for recognition, classification, reasoning, prediction, control / decision-making, etc.

[0005] Recently, in order to speed up the operation of artificial intelligence (AI), neural processing units (NPUs) are being developed.

[0006] On the other hand, a system configured by various semiconductor components is implemented by a board-level system based on a printed circuit board (PCB).

[0007] Since high integration can be achieved according to the development of semiconductor manufacturing process technology, a system on chip (SoC) or a system in a package (SiP) is being proposed, in which various semiconductor components such as a processor, a memory, and peripheral devices are implemented in one chip, and in which various semiconductor components such as a processor, a memory, and peripheral devices are implemented in one package.

[0008] SoC refers to a semiconductor device (chip) that contains the entire system in one chip, and refers to a technology that implements major semiconductor components such as operation, memory, and data conversion elements in one chip. SiP refers to a semiconductor device that contains the entire system in one package, and refers to a technology that implements major semiconductor components such as operation, memory, and data conversion elements in one package. That is, the central processing unit (CPU), digital signal processing (DSP) chip, microcontroller unit (MCU), etc. of a computer are integrated in one semiconductor die or package, so that the chip or package itself is used as a system. As mentioned above, when semiconductor devices with multiple functions are combined in one chip, the board space is significantly reduced, so that the size of various electronic products can be reduced. In addition, compared with the technology of manufacturing multiple semiconductor devices separately, the manufacturing cost of semiconductor devices is significantly reduced, thereby also reducing the sales unit price of electronic products. Therefore, SoC or SiP technology that integrates all component functions is becoming a core component technology in the advanced digital era, which focuses on high performance, low cost and small size. Summary of the invention

[0009] The inventors of the present disclosure have recognized that when an NPU is integrated in a SoC or a SiP, the size of a board substrate is reduced, thereby making it possible to innovatively reduce the size of an electronic device.

[0010] Furthermore, the inventors of the present disclosure have recognized that when an NPU is integrated in a SoC or a SiP, manufacturing costs can be reduced compared to a separately manufactured semiconductor device.

[0011] However, the inventors of the present disclosure also recognize that when multiple semiconductor components are integrated in a SoC or SiP, the complexity increases significantly, which increases the defect rate in the manufacturing process. Defects in the manufacturing process may be discovered in pre-factory testing, but minor defects in some components integrated in the SoC or SiP are not discovered in pre-factory testing and may be handed over to users. Such minor defects are gradually amplified due to fatigue stress or physical stress caused by repeated use, ultimately leading to erroneous operation of the SoC or SiP.

[0012] When a SoC or SiP is installed in an electronic device for user entertainment, its erroneous operation may not be so problematic. However, the inventors of the present disclosure have recognized that the situation is different when a SoC or SiP is installed in a mission-critical product.

[0013] Specifically, the inventors of the present disclosure have recognized a problem in which unpredictable artificial intelligence (AI) operation results may be output when an NPU in a SoC or SiP erroneously operates due to a malfunction, defect, or damage.

[0014] For example, the inventors of the present disclosure have recognized that when a SoC or SiP including an NPU is used in electronic devices installed in autonomous vehicles, drones, urban air mobility (UAM), unmanned aerial vehicles (UAV), or in electronic devices installed in AI robots, unpredictable AI operation results may be output due to NPU failure, defect, or damage.

[0015] Therefore, the inventors of the present disclosure have recognized the need for a method of performing tests in a SoC or SiP at runtime, which tests have heretofore been performed only before shipment.

[0016] In other words, it has been recognized that there is a need to detect defects through testing.

[0017] According to one aspect of the present disclosure, a system is provided that is capable of testing components in the system during runtime. The system may include: a substrate; a plurality of functional components mounted on the substrate and including circuits; a system bus formed on the substrate using a conductive pattern to allow the plurality of functional components to communicate with each other; one or more wrappers, each of the one or more wrappers connected to one of the plurality of functional components; and an in-system component tester (ICT) configured to: select at least one functional component in an idle state among the plurality of functional components as a component under test (CUT); and test the at least one functional component selected as the CUT through the one or more test wrappers.

[0018] The ICT may be further configured to monitor states of the plurality of functional components via the one or more test wrappers.

[0019] The ICT may also be configured to: stop the test on the at least one functional component selected as the CUT based on detecting a conflict with access from the system bus to the at least one functional component selected as the CUT; and based on the stop, allow the connection of the at least one functional component to return to the system bus.

[0020] The ICT may also be configured to return to the selecting step if the at least one functional component is in the idle state again after the connection of the at least one functional component is returned to the system bus.

[0021] Returning to the selecting step may occur after expiration of a back-off timer with respect to the conflict.

[0022] The plurality of functional components include one or more universal processing units (UPUs).

[0023] The UPU may include at least one or more of the following: one or more central processing units (CPUs); one or more graphics processing units (GPUs); and one or more neural processing units (NPUs) configured to perform operations of an artificial neural network (ANN) model.

[0024] The plurality of functional components also include at least one or more of: at least one memory; at least one memory controller; and at least one input and output (I / O) controller.

[0025] The ICT may also be configured to, for the testing step, instruct the one or more wrappers to isolate a connection of the at least one functional component selected as the CUT from the system bus.

[0026] The ICT may include at least one or more of the following: a detector configured to monitor the status of multiple functional components; a scheduler configured to manage the operation of the ICT; a generator configured to generate test input data; and a tester configured to import the test input data into the CUT and analyze the test results obtained from the CUT that processes the test input data.

[0027] The test input data is predefined test data or a random bit stream generated based on a seed.

[0028] The ICT may be further configured to: analyze the test results obtained from the at least one functional component selected as the CUT after the test step is completed; and allow the at least one functional component to be connected to the system bus or another system connection based on the at least one functional component being analyzed as normal.

[0029] The ICT may also be configured to deactivate the at least one functional component based on the at least one functional component being analyzed as defective.

[0030] The system may further include a field programmable gate array (FPGA) configured to emulate the at least one functional component analyzed as defective.

[0031] The address of the FPGA can be deactivated and replaced by the address of the at least one functional component analyzed as defective.

[0032] The deactivating step may include revoking the address of the at least one functional component analyzed as defective.

[0033] The deactivating step may include powering off the at least one functional component analyzed as being defective.

[0034] The deactivating step may include isolating the at least one functional component analyzed as defective from the system bus by cutting off a system bus connection with the at least one functional component analyzed as defective.

[0035] The plurality of functional components include a spare component for the at least one functional component analyzed as defective. The ICT may also be configured to enable the spare component.

[0036] The test step may include a scan test different from a functional test. For the scan test, the ICT may also be configured to: interconnect a plurality of triggers in each CUT; introduce a test input into at least one trigger; and obtain a test result from the operation of the combinational logic of the trigger to analyze whether the CUT is defective or normal during run time.

[0037] The plurality of functional components may include a neural processing unit (NPU). The NPU may include a plurality of processing element arrays and may be configured to select and test at least one processing element in the plurality of processing element arrays.

[0038] Although the present disclosure mainly describes SoC, the present disclosure is not limited to SoC but may be applied to a system-in-package (SIP) or printed circuit board (PCB) based board-level system. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The above and other aspects, features and other advantages of the present disclosure will be more clearly understood from the following detailed description taken in conjunction with the accompanying drawings, in which:

[0040] Figure 1 is a schematic conceptual diagram illustrating a neural processing unit according to the present disclosure;

[0041] Figure 2 is a schematic conceptual diagram illustrating one processing element of a processing element array suitable for use with the present disclosure;

[0042] Figure 3 It is shown Figure 1 An example diagram of a modified example of a neural processing unit 100;

[0043] Figure 4 is a schematic conceptual diagram illustrating an exemplary artificial neural network model;

[0044] Figure 5A It is a view showing the basic structure of a convolutional neural network;

[0045] Figure 5B It is a view showing the overall action of the convolutional neural network.

[0046] Fig. 6A It is shown that Figure 1 or 3 ;

[0047] Figure 6B It is shown Fig. 6A A view of a first modified example of;

[0048] Figure 6C It is shown Fig. 6A A view of a second modified example of;

[0049] Fig.6D It is shown that Figure 1 or Figure 3 A diagram of an exemplary architecture of a computing system of an NPU as shown in;

[0050] Fig. 6E It is shown Fig.6D A view of a first modified example of;

[0051] Fig. 6F It is shown Fig.6D A view of a second modified example of;

[0052] Figure 7 is a diagram showing an example of a scan trigger;

[0053] Figure 8 is a diagram showing an example of adding an architecture for scan testing in hardware design;

[0054] Fig. 9A From an operational point of view, Figures 6A to 6C SoC or Figures 6D to 6F Example diagram of a system;

[0055] Fig. 9B is an example diagram showing a configuration for testing an NPU;

[0056] Fig.10 is a schematic diagram illustrating the operation of the wrapper;

[0057] Fig.11 is a schematic diagram showing the internal configuration of the ICT;

[0058] Fig.12 is a block diagram specifically illustrating the operation of an ICT monitoring function component to determine whether the function component is in an idle state;

[0059] Fig.13 is an example diagram showing operations between a master, a slave, and an arbiter operating on a system bus;

[0060] Fig.14 is a diagram showing an example of adding a shift register in a SoC chip;

[0061] Fig.15 is an example diagram showing the operation sequence of the ICT;

[0062] Fig.16 is a block diagram showing a test process of an internal memory for ease of understanding;

[0063] Fig.17 is an example diagram showing a process of testing a function using a random number generator;

[0064] Fig.18A is a diagram showing an example of a plurality of clocks, Fig.18B is an example diagram showing the operation of a tester at multiple clocks, Fig. 18C is a view showing the path of test input data;

[0065] Fig.19A is a view showing an example of a functional component, Fig.19B is a diagram showing an example of test input data (eg, test vectors) imported into a tester in the ICT;

[0066] Fig. 20 is a diagram showing a test process using DFT;

[0067] Fig.21 is a diagram showing an example of shift data and captured data during a test;

[0068] Fig. 22 is a view showing an example of switching the test mode to the normal operation mode;

[0069] Fig.23 is a diagram showing an example of a flip-flop operating on a scan chain;

[0070] Fig.24 is a view showing a portion of the CUT operating in a normal operation mode;

[0071] Fig.25 is a diagram showing an example of a simulated process; and

[0072] Fig.26 is a diagram showing a test architecture for a JPEG image encoder. DETAILED DESCRIPTION

[0073] The specific structure or step-by-step description of the embodiments of the concepts of the present disclosure disclosed in this specification or application is only for the purpose of illustrating the embodiments of the concepts of the present disclosure. The examples of the concepts of the present disclosure can be implemented in various forms and are not interpreted as being limited to the examples described in this specification or application.

[0074] Various modifications and changes can be applied to the examples according to the concept of the present disclosure, and the examples can have various forms, so the examples will be described in detail in the specification or application with reference to the drawings. However, it should be understood that the examples according to the concept of the present disclosure are not limited to specific examples, but include all changes, equivalents or alternatives included in the spirit and technical scope of the present disclosure.

[0075] Terms such as first and / or second can be used to describe various components, but the components are not limited to the above terms. The above terms are used to distinguish one component from another component. For example, without departing from the scope of the present invention, a first component can be called a second component, and similarly, a second component can be called a first component.

[0076] It should be understood that when an element is described as being "coupled" or "connected" to another element, the element may be directly coupled or directly connected to the other element, or coupled or connected to the other element through a third element. Conversely, when an element is described as being "directly coupled" or "directly connected" to another element, it should be understood that there are no elements between them. Other expressions describing the relationship between components, such as "between," "adjacent," and "directly adjacent," should be interpreted in the same manner.

[0077] The terms used in this specification are only used to describe specific examples and are not intended to limit the present disclosure. If there is no obvious contrary meaning in the context, the singular form may include the plural form. In this specification, it should be understood that the term "including" or "having" refers to the presence of the features, quantities, steps, operations, components, parts or combinations thereof described in the specification, but does not exclude the possibility of pre-existing or adding one or more other features, quantities, steps, operations, components, parts or combinations thereof.

[0078] If not defined to the contrary, all terms used herein, including technical or scientific terms, have the same meanings as those generally understood by those of ordinary skill in the art. Terms defined in general dictionaries should be interpreted as having the same meanings as in the context of the relevant technology, but if not clearly defined in this specification, they should not be interpreted as ideal or overly formal meanings.

[0079] When describing examples, technologies that are well known in the technical field of the present disclosure and have no direct relationship with the present disclosure will not be described. The reason is that unnecessary descriptions are omitted to clearly convey the main points of the present disclosure without obscuring the main points.

[0080] <Definition of terms>

[0081] Here, in order to help understanding of the disclosure set forth in this specification, the terms used in this specification will be briefly defined.

[0082] NPU is the abbreviation of Neural Processing Unit, which refers to a processor dedicated to the operation of artificial neural network models separate from the central processing unit (CPU).

[0083] ANN is the abbreviation of artificial neural network, which refers to a network that imitates human intelligence by simulating the connection of neurons in the human brain through synapses and connecting nodes in a hierarchical structure.

[0084] The information about the structure of the artificial neural network includes information about the number of layers, the number of nodes in a layer, the value of each node, information about an operation processing method, and information about a weight matrix applied to each node.

[0085] The information about the data locality of the artificial neural network is information predicting the order of operations of the artificial neural network model processed by the neural processing unit based on the order of data access requests requested by the neural processing unit to the individual memories.

[0086] DNN is the abbreviation of Deep Neural Network, which may mean increasing the number of hidden layers of artificial neural networks to achieve higher artificial intelligence.

[0087] CNN, short for Convolutional Neural Network, is a type of neural network that functions similarly to image processing performed in the visual cortex of the human brain. Convolutional neural networks are known to be suitable for image processing and are known to be easy to extract features of input data and recognize patterns of features.

[0088] A kernel refers to the weight matrix applied to a CNN.

[0089] Hereinafter, the present disclosure will be described in detail by explaining examples of the present disclosure with reference to the attached drawings.

[0090] Figure 1 A neural processing unit according to the present disclosure is illustrated.

[0091] Figure 1 The neural processing unit (NPU) 100 shown is a processor specialized for performing operations for artificial neural networks.

[0092] An artificial neural network is a network in which artificial neurons are assembled, and when there are various inputs or incoming stimuli, it multiplies the weights by the inputs or stimuli, adds the products, and then uses an activation function to convert the values ​​after adding the bias for transmission. The artificial neural network trained as described above can be used to output inference results based on the input data.

[0093] The neural processing unit 100 may be a semiconductor device implemented by an electrical / electronic circuit. An electrical / electronic circuit may refer to a circuit including a large number of electronic components (transistors, capacitors, etc.). The neural processing unit 100 includes a processing element (PE) array 110, an NPU internal memory 120, an NPU scheduler 130, and an NPU interface 140. Each of the processing element array 110, the NPU internal memory 120, the NPU scheduler 130, and the NPU interface 140 may be a semiconductor circuit connected to a large number of electronic components. Therefore, some electronic components may be difficult to identify or distinguish with the naked eye, but can only be identified through operation. For example, any circuit may operate as a processing element array 110 or as an NPU scheduler 130.

[0094] The neural processing unit 100 may include a processing element array 110, an NPU internal memory 120 configured to store an artificial neural network model inferred from the processing element array 110, and an NPU scheduler 130 configured to control the processing element array 110 and the NPU internal memory 120 based on data locality information of the artificial neural network model or information about its structure. Here, the artificial neural network model may include data locality information of the artificial neural network or information about its structure. The artificial neural network model may refer to an AI recognition model that has been trained to perform a specific reasoning function.

[0095] The processing element array 110 may perform operations of an artificial neural network. For example, when input data is input, the processing element array 110 may enable the artificial neural network to perform learning. When input data is input after learning is completed, the processing element array 110 may perform operations of inferring inference results through the artificial neural network that has completed learning.

[0096] The NPU interface 140 can communicate with the Figure 5A , 6A Or 6B's ANN drives communication with various components (e.g., memory) in the device.

[0097] For example, the neural processing unit 100 can use the NPU interface 140 to store Figure 5A , 6A Or the data of the artificial neural network model in the memory 200 of 6B is called to the NPU internal memory 120.

[0098] The NPU scheduler 130 is configured to control the operation of the processing element array 110 and the read / write instructions of the NPU internal memory 120 to perform inference operations of the neural processing unit 100.

[0099] The NPU scheduler 130 may be configured to analyze data locality information of the artificial neural network model or information about its structure to control the processing element array 110 and the NPU internal memory 120 .

[0100] The NPU scheduler 130 may analyze or receive the structure of an artificial neural network model that may be operated in the processing element array 110. The data of the artificial neural network that may be included in the artificial neural network model may store node data of each layer, placement data locality information of the layer or information about the structure, and weight data of each connection network connecting the nodes of the layer. The data of the artificial neural network may be stored in a memory provided in the NPU scheduler 130 or the NPU internal memory 120. The NPU scheduler 130 may access Figure 5A , 6A or 6B memory 200 to utilize the necessary data. However, it is not limited to this, so that data locality information of the artificial neural network model or information about its structure can be generated based on data such as node data and weight data of the artificial neural network model. Weight data can also be called weight kernel. Node data is also called feature map. For example, data defining the structure of the artificial neural network model can be generated when designing the artificial neural network model or completing learning, but the present disclosure is not limited to this.

[0101] The NPU scheduler 130 may schedule the operation order of the artificial neural network model based on data locality information of the artificial neural network model or information about its structure.

[0102] The NPU scheduler 130 can obtain the memory address value in which the node data of the layer of the artificial neural network model and the weight data of the connection network are stored based on the data locality information of the artificial neural network model or the information about its structure. For example, the NPU scheduler 130 can obtain the memory address value in which the node data of the layer of the artificial neural network model stored in the memory and the weight data of the connection network are stored. Therefore, the NPU scheduler 130 can obtain the node data of the layer of the artificial neural network model to be driven and the weight data of the connection network from the memory 200 to store the obtained data in the NPU internal memory 120. The node data of each layer may have a corresponding memory address value. The weight data of each connection network may have a corresponding memory address value.

[0103] The NPU scheduler 130 may schedule the operation order of the processing unit array 110 based on data locality information of the artificial neural network model or information about its structure (e.g., placement of layers of the artificial neural network, data locality information of the artificial neural network model or information about its structure).

[0104] The NPU scheduler 130 performs scheduling based on data locality information of the artificial neural network model or information about its structure, so that the NPU scheduler can operate in a different manner from the scheduling concept of the ordinary CPU. The scheduling of the ordinary CPU operates to provide the highest efficiency in consideration of fairness, efficiency, stability, and reaction time. That is, in consideration of priority and operation time, the ordinary CPU schedules to perform the most processing at the same time.

[0105] Known CPUs use algorithms that schedule tasks in consideration of data such as priority of each process or operation processing time. In contrast, the NPU scheduler 130 may determine the processing order based on data locality information of an artificial neural network model or information about its structure.

[0106] Furthermore, the NPU scheduler 130 may determine the processing order based on data locality information of the artificial neural network model or information about its structure and / or data locality information of the neural processing unit 100 to be used or information about its structure.

[0107] However, the present disclosure is not limited to the data locality information of the neural processing unit 100 or the information about its structure. For example, the data locality information of the neural processing unit 100 or the information about its structure can determine the processing order by utilizing at least one of the memory size of the NPU internal memory 120, the hierarchical structure of the NPU internal memory 120, the number (size) data of the processing elements PE1 to PE12, and the operator structure of the processing elements PE1 to PE12. That is, the data locality information of the neural processing unit 100 or the information about its structure may include the memory size of the NPU internal memory 120, the hierarchical structure of the NPU internal memory 120, the number data of the processing elements PE1 to PE12, and the operator structure of the processing elements PE1 to PE12. However, the present disclosure is not limited to the data locality information of the neural processing unit 100 or the information about its structure. The memory size of the NPU internal memory 120 includes information about the memory capacity. The hierarchical structure of the NPU internal memory 120 includes information about the connection relationship between specific layers of each hierarchical structure. The operator structures of processing elements PE1 to PE12 include information about components in the processing elements.

[0108] The neural processing unit 100 according to an example of the present disclosure may include at least one processing element, an NPU internal memory 120 storing an artificial neural network model inferred from the at least one processing element, and an NPU scheduler 130, which is configured to control the at least one processing element and the NPU internal memory 120 based on data locality information of the artificial neural network model or information about its structure. The NPU scheduler 130 may be configured to be further provided with data locality information of the neural processing unit 100 or information about its structure. In addition, the data locality information of the neural processing unit 100 or information about its structure may include at least one of the memory size of the NPU internal memory 120, the hierarchical structure of the NPU internal memory 120, the number (size) data of at least one processing unit, and the operator structure of at least one processing unit.

[0109] According to the structure of the artificial neural network model, the operations of each layer are performed in sequence. That is to say, when the structure of the artificial neural network model is determined, the order of operations of each layer can be determined. According to the structure of the artificial neural network model, the order of operations or the order of data flow can be defined as the data locality of the artificial neural network model at the algorithm level.

[0110] When the compiler compiles an artificial neural network model to be executed in the neural processing unit 100, the artificial neural network data locality of the artificial neural network model at the neural processing unit-memory level can be reconstructed.

[0111] That is, the data locality of the artificial neural network model at the neural processing unit-memory level can be constructed based on the compiler, the algorithm applied to the artificial neural network model, and the operating characteristics of the neural processing unit 100.

[0112] For example, even in the same artificial neural network model, the artificial neural network data locality of the artificial neural network model to be processed may be configured differently depending on the method of operating the artificial neural network model through the neural processing unit 100, such as feature map tiling or static technology of the processing element, the number of processing elements of the neural processing unit 100, the cache memory capacity such as feature map or weight in the neural processing unit 100, the memory hierarchy structure in the neural processing unit 100, and the algorithm characteristics of the compiler that determines the operation order of the neural processing unit 100 to operate the artificial neural network model. This is because even if the same artificial neural network model is calculated by the above-mentioned factors, the neural processing unit 100 may determine the order of required data differently at each moment in units of clock cycles.

[0113] The compiler can determine the order of data required for physical operation processing by constructing the artificial neural network model at the neural processing unit-memory level in units of words for artificial neural network data locality.

[0114] In other words, the artificial neural network data locality of the artificial neural network model existing at the neural processing unit-memory level can be defined as information that predicts the operation order of the artificial neural network model processed by the neural processing unit 100 based on the order of data access requests requested by the neural processing unit 100 to the memory 200.

[0115] The NPU scheduler 130 may be configured to store data locality information of the artificial neural network or information about its structure.

[0116] That is, even if only the data locality information of the artificial neural network of the artificial neural network model or the information about its structure is used, the NPU scheduler 130 can determine the processing order (sequence). That is, the NPU scheduler 130 can determine the operation order by using the data locality information or the information about the structure from the input layer to the output layer of the artificial neural network. For example, the input layer operation can be scheduled first, and the output layer operation can be scheduled last. Therefore, when the NPU scheduler 130 is provided with the data locality information of the artificial neural network model or the information about its structure, all operation sequences of the artificial neural network model can be known. Therefore, all scheduling orders can be determined.

[0117] Furthermore, the NPU scheduler 130 may determine a processing order by considering data locality information of the artificial neural network model or information about its structure and data locality information of the neural processing unit 100 or information about its structure and optimize processing for each determined order.

[0118] Therefore, when the data locality information or information about the structure of all artificial neural network models and the data locality information or information about the structure of the neural processing unit 100 are provided to the NPU scheduler 130, the operation efficiency of each scheduling order determined by the data locality information or information about the structure of the artificial neural network model can be further improved. For example, the NPU scheduler 130 can obtain connection network data having four artificial neural network layers and weight data of three layers connecting these layers. In this case, the method of scheduling the processing order by the NPU scheduler 130 based on the data locality information or information about the structure of the artificial neural network model will be described below by way of example.

[0119] For example, the NPU scheduler 130 may set the input data for the reasoning operation to the node data of the first layer as the input layer of the artificial neural network model, and schedule to first perform the multiplication and accumulation (MAC) operation of the node data of the first layer and the weight data of the first connection network corresponding to the first layer. However, the examples of the present disclosure are not limited to MAC operations and artificial neural network operations may be performed by utilizing multipliers and adders that may be modified in various forms. In the following, for the convenience of description, the corresponding operation is referred to as the first operation, the result of the first operation is referred to as the first operation value, and the corresponding schedule may be referred to as the first schedule.

[0120] For example, the NPU scheduler 130 may set the first operation value to the node data of the second layer corresponding to the first connection network after the first scheduling, and schedule to perform a MAC operation of the node data of the second layer and the weight data of the second connection network corresponding to the second layer. Hereinafter, for the convenience of description, the corresponding operation is referred to as the second operation, the result of the second operation is referred to as the second operation value, and the corresponding scheduling may be referred to as the second scheduling.

[0121] For example, the NPU scheduler 130 may set the second operation value to the node data of the third layer corresponding to the second connection network during the second scheduling, and schedule to perform a MAC operation of the node data of the third layer and the weight data of the third connection network corresponding to the third layer. Hereinafter, for the convenience of description, the corresponding operation is referred to as the third operation, the result of the third operation is referred to as the third operation value, and the corresponding scheduling is referred to as the third scheduling.

[0122] For example, the NPU scheduler 130 may set the third operation value as the node data of the fourth layer, which is the output layer corresponding to the third connection network, and schedule to store the inference result stored in the node data of the fourth layer in the NPU internal memory 120. Hereinafter, for the convenience of description, the corresponding schedule may be referred to as the fourth schedule.

[0123] In summary, the NPU scheduler 130 can control the NPU internal memory 120 and the processing element array 110 to operate in the order of the first schedule, the second schedule, the third schedule, and the fourth schedule. That is, the NPU scheduler 130 can be configured to control the NPU internal memory 120 and the processing element array 110 to perform operations according to the set scheduling order.

[0124] In summary, the neural processing unit 100 according to the example of the present disclosure may be configured to schedule a processing order based on the structure of the layers of the artificial neural network and the operation order data corresponding to the structure.

[0125] For example, the NPU scheduler 130 may be configured to schedule a processing order from an input layer to an output layer of an artificial neural network of an artificial neural network model based on data locality information or information about a structure.

[0126] The NPU scheduler 130 may improve the operation rate and memory reusability of the neural processing unit by controlling the NPU internal memory 120 by utilizing the scheduling order based on the data locality information of the artificial neural network model or information about its structure.

[0127] According to the characteristics of the artificial neural network operation driven in the neural processing unit 100 according to the example of the present disclosure, the operation value of one layer can be used as input data of a subsequent layer.

[0128] Therefore, the neural processing unit 100 controls the NPU internal memory 120 according to the scheduling order to improve the memory reuse of the NPU internal memory 120. The reuse of the memory can be determined by the number of times the data stored in the memory is read. For example, after storing specific data in the memory, when the specific data is read only once and then deleted or overwritten, the memory reuse rate may become 100%. For example, after storing specific data in the memory, when the specific data is read four times and then deleted or overwritten, the memory reuse rate may become 400%. That is, the memory reuse rate can be determined as the number of times the stored data is reused. That is, the memory reuse rate can refer to the reuse of data stored in the memory or a specific memory address storing specific data.

[0129] Specifically, when the NPU scheduler 130 is configured to be provided with data locality information of an artificial neural network model or information about its structure and to calculate sequential data in which operations of the artificial neural network are performed by the provided data locality information of the artificial neural network model or information about its structure, the NPU scheduler 130 identifies the calculation results of the node data of a specific layer of the artificial neural network model and the weight data of a specific connection network as the node data of the corresponding subsequent layer.

[0130] Therefore, the NPU scheduler 130 can reuse the value of the memory address storing the specific operation result for subsequent operations, thereby improving the reuse rate of the memory.

[0131] For example, the first operation value of the first schedule is set as the node data of the second layer of the second schedule. More specifically, the NPU scheduler 130 can reset the memory address value corresponding to the first operation value of the first schedule stored in the NPU internal memory 120 to the memory address value corresponding to the node data of the second layer of the second schedule. That is, the memory address value can be reused. Therefore, the NPU scheduler 130 reuses the data of the memory address of the first schedule, so that the NPU internal memory 120 can utilize the second layer node data as the second schedule without a separate memory write operation.

[0132] For example, the second operation value of the second schedule is set as the node data of the third layer of the third schedule. More specifically, the NPU scheduler 130 can reset the memory address value corresponding to the second operation value of the second schedule stored in the NPU internal memory 120 to the memory address value corresponding to the node data of the third layer of the third schedule. That is, the memory address value can be reused. Therefore, the NPU scheduler 130 reuses the data of the memory address of the second schedule, so that the NPU internal memory 120 can use the data as the third layer node data of the third schedule without a separate memory write operation.

[0133] For example, the third operation value of the third schedule is set as the node data of the fourth layer of the fourth schedule. More specifically, the NPU scheduler 130 can reset the memory address value corresponding to the third operation value of the third schedule stored in the NPU internal memory 120 to the memory address value corresponding to the node data of the fourth layer of the fourth schedule. That is, the memory address value can be reused. Accordingly, the NPU scheduler 130 reuses the data of the memory address of the third schedule so that the NPU internal memory 120 can use the data as the fourth layer node data of the fourth schedule without a separate memory write operation.

[0134] In addition, the NPU scheduler 130 may be configured to determine whether to reuse the scheduling order and memory to control the NPU internal memory 120. In this case, the NPU scheduler 130 analyzes data locality information of the artificial neural network model or information about its structure to provide effective scheduling. In addition, data required for operations that can reuse the memory is not repeatedly stored in the NPU internal memory 120, so that the memory usage can be reduced. In addition, the NPU scheduler 130 can improve the efficiency of the NPU internal memory 120 by calculating the memory usage that is reduced as much as the memory is reused.

[0135] In addition, the NPU scheduler 130 may be configured to monitor resource usage of the NPU internal memory 120 and resource usage of the processing elements PE1 to PE12 based on data locality information or information about the structure of the neural processing unit 100. Therefore, the hardware resource utilization efficiency of the neural processing unit 100 may be improved.

[0136] The NPU scheduler 130 of the neural processing unit 100 according to an example of the present disclosure may have the effect of reusing memory by utilizing data locality information of an artificial neural network model or information about its structure.

[0137] In other words, when the artificial neural network model is a deep neural network, the number of layers and the number of connected networks may increase significantly, so that in this case, the effect of memory reuse can be maximized to a greater extent.

[0138] That is, when the neural processing unit 100 does not calculate the data locality information of the artificial neural network model or the information about the structure and operation order, the NPU scheduler 130 may not determine whether to reuse the value stored in the NPU internal memory 120. Therefore, the NPU scheduler 130 unnecessarily generates the memory address required for each processing and copies substantially the same data from one memory address to another memory address. Therefore, unnecessary memory read and write tasks are generated, and duplicate values ​​are stored in the NPU internal memory 120, which may lead to the problem that the memory is wasted unnecessarily.

[0139] The processing element array 110 refers to a configuration in which a plurality of processing elements PE1 to PE12 are provided, and the plurality of processing elements PE1 to PE12 are configured to operate node data of an artificial neural network and weight data of a connection network. Each processing element may include a multiplication and accumulation (MAC) operator and / or an arithmetic logic unit (ALU) operator, but examples according to the present disclosure are not limited thereto.

[0140] although Figure 2 A plurality of processing units are shown as an example, but operators implemented by a plurality of multiplier and adder trees may also be configured to be arranged in parallel in one processing unit rather than in a MAC. In this case, the processing element array 110 may also be referred to as at least one processing element including a plurality of operators.

[0141] Processing element array 110 is configured to include a plurality of processing elements PE1 to PE12 . Figure 2The plurality of processing elements PE1 to PE12 is merely an example for convenience of description, and the number of the plurality of processing elements PE1 to PE12 is not limited. The size or number of processing element array 110 may be determined by the number of the plurality of processing elements PE1 to PE12. The size of processing element array 110 may be implemented by an N×M matrix. Here, N and M are integers greater than zero. Processing element array 110 may include N×M processing elements. That is, one or more processing elements may be provided.

[0142] The size of the processing element array 110 may be designed in consideration of the characteristics of the artificial neural network model in which the neural processing unit 100 operates. As an additional explanation, the number of processing elements may be determined in consideration of the data size, the required operating speed, and the required power consumption at which the artificial neural network model operates. The data size of the artificial neural network model may be determined so as to correspond to the number of layers of the artificial neural network model and the weight data size of each layer.

[0143] Therefore, the size of the processing element array 110 of the neural processing unit 100 according to the example of the present disclosure is not limited. As the number of processing elements of the processing element array 110 increases, the parallel computing power of the artificial neural network model operating increases, but the manufacturing cost and physical size of the neural processing unit 100 may increase.

[0144] For example, the artificial neural network model running in the neural processing unit 100 may be an artificial neural network trained to detect thirty specific keywords, i.e., an AI keyword recognition model. In this case, the size of the processing element array 110 of the neural processing unit 100 may be designed to be 4×3, taking into account the characteristics of the operation volume. In other words, the neural processing unit 100 may include twelve processing elements. However, not limited to this, the number of multiple processing elements PE1 to PE12 may be selected within the range of 8 to 16,384. That is, the examples of the present disclosure are not limited to the number of processing elements.

[0145] The processing element array 110 is configured to perform functions such as addition, multiplication, and accumulation required for artificial neural network operations. In other words, the processing element array 110 may be configured to perform multiplication and accumulation (MAC) operations.

[0146] In the following, the first processing element PE1 in the processing element array 110 is taken as an example for description.

[0147] Figure 2 One processing element of a processing element array suitable for use with the present disclosure is illustrated.

[0148] The neural processing unit 100 according to an example of the present disclosure may include a processing element array 110, an NPU internal memory 120 configured to store an artificial neural network model inferred from the processing element array 110, and an NPU scheduler 130 configured to control the processing element array 110 and the NPU internal memory 120 based on data locality information of the artificial neural network model or information about its structure. The processing element array 110 is configured to perform a MAC operation, and the processing element array 110 is configured to quantize and output a MAC operation result, but the embodiments of the present disclosure are not limited thereto.

[0149] The NPU internal memory 120 can store all or part of the artificial neural network model according to the memory size and data size of the artificial neural network model.

[0150] The first processing element PE1 may include a multiplier 111, an adder 112, an accumulator 113, and a bit quantization unit 114. However, examples according to the present disclosure are not limited thereto, and the processing element array 110 may be modified in consideration of operation characteristics of an artificial neural network.

[0151] The multiplier 111 multiplies the input (N)-bit data and (M)-bit data. The operation value of the multiplier 111 is output as (N+M)-bit data. Here, N and M are integers greater than zero. The first input unit receiving the (N)-bit data may be configured to receive a value having a variable characteristic, and the second input unit receiving the (M)-bit data may be configured to receive a value having a constant characteristic. When the NPU scheduler 130 distinguishes between variable value and constant value characteristics, the NPU scheduler 130 can increase the memory reusability of the NPU internal memory 120. However, the input data of the multiplier 111 is not limited to constant values ​​and variable values. That is, according to the example of the present disclosure, the input data of the processing element can be operated by understanding the characteristics of the constant value and the variable value, so that the operating efficiency of the neural processing unit 100 can be improved. However, the neural processing unit 100 is not limited to the characteristics of the constant value and the variable value of the input data.

[0152] Here, the value having the characteristics of a variable or the variable means that the value of the memory address storing the corresponding value is updated whenever the input data input is updated. For example, the node data of each layer can be a MAC operation value reflecting the weight data of the artificial neural network model. When object recognition of motion image data or the like is inferred by the artificial neural network model, the input image changes at each frame, thereby changing the node data of each layer.

[0153] Here, regardless of the update of the input data input, a value having a constant characteristic is stored, wherein the constant may refer to the value of a memory address where a corresponding value is stored. For example, even if object recognition of motion image data or the like is inferred by an artificial neural network model with a unique inference determination standard of the artificial neural network model, the weight data of the connection network may not be changed.

[0154] That is, the multiplier 111 can be configured to receive a variable and a constant. As an additional explanation, the variable value input to the first input unit can be the node data of the layer of the artificial neural network, and the node data can be the input data of the input layer of the artificial neural network, the accumulated value of the hidden layer, and the accumulated value of the output layer. The constant value input to the second input unit can be the weight data of the connection network of the artificial neural network.

[0155] The NPU scheduler 130 may be configured to improve memory reuse rate by taking into account characteristics of constant values.

[0156] The variable value is an operation value of each layer, and the NPU scheduler 130 may identify reusable variable values ​​based on data locality information of the artificial neural network model or information about its structure, and control the NPU internal memory 120 to reuse the memory.

[0157] The constant value is weight data of each connection network, and the NPU scheduler 130 can identify the constant value of the reused connection network based on data locality information of the artificial neural network model or information about its structure, and control the NPU internal memory 120 to reuse the memory.

[0158] That is, the NPU scheduler 130 may be configured to identify reusable variable values ​​and reusable constant values ​​based on data locality information of the artificial neural network model or information about its structure, and control the NPU internal memory 120 to reuse the memory.

[0159] The processing element knows that when 0 is input to one of the first input unit and the second input unit of the multiplier 111, even if no operation is performed, the operation result is 0. Therefore, the processing element can limit the operation of the multiplier 111 so as not to perform the operation.

[0160] For example, when 0 is input to one of the first input unit and the second input unit of the multiplier 111 , the multiplier 111 may be configured to operate in a zero skip method.

[0161] The bit width of the data input to the first input unit and the second input unit can be determined according to the quantization of the node data and the weight data of each layer in the artificial neural network model. For example, when the node data of the first layer can be quantized to five bits and the weight data of the first layer can be quantized to seven bits. In this case, the first input unit can be configured to receive five-bit data and the second input unit can be configured to receive seven-bit data.

[0162] When the quantized data stored in the NPU internal memory 120 is input to the input unit of the processing element, the neural processing unit 100 can control the quantization bit width to be converted in real time. That is, each layer can have a different quantization bit width, and when the bit width of the input data is converted, the processing element can be configured to receive the bit width information from the neural processing unit 100 in real time and convert the bit width in real time to generate the input data.

[0163] The accumulator 113 accumulates the operation value of the multiplier 111 and the operation value of the accumulator 113 as many times as (L) cycles using the adder 112. Therefore, the bit width of the data of the output unit and the input unit of the accumulator 113 can be output to (N+M+log2(L)) bits. Here, L is an integer greater than zero.

[0164] When the accumulation is completed, the accumulator 113 is applied with an initialization reset to initialize the data stored in the accumulator 113 to zero, but examples according to the present disclosure are not limited thereto.

[0165] The bit quantization unit 114 can reduce the bit width of the data output from the accumulator 113. The bit quantization unit 114 can be controlled by the NPU scheduler 130. The bit width of the quantized data can be output to (X) bits. Here, X is an integer greater than zero. According to the above configuration, the processing element array 110 is configured to perform a MAC operation, and the processing element array 110 can quantize the MAC operation result to output the result. Quantization may have the effect that the larger the number of (L) cycles, the lower the power consumption. In addition, when power consumption is reduced, heat generation can also be reduced. Specifically, when heat generation is reduced, the possibility of malfunction of the neural processing unit 100 due to high temperature can be reduced.

[0166] The output data (X) bits of the bit quantization unit 114 can be used as node data of a subsequent layer or input data of a convolution. When the artificial neural network model is quantized, the bit quantization unit 114 can be configured to be provided with quantization information from the artificial neural network model. However, not limited to this, the NPU scheduler 130 can also be configured to extract quantization information by analyzing the artificial neural network model. Therefore, the output data (X) bits are converted into a quantization bit width to be output to correspond to the quantized data size. The output data (X) bits of the bit quantization unit 114 can be stored in the NPU internal memory 120 with a quantized bit width.

[0167] The processing element array 110 of the neural processing unit 100 according to the example of the present disclosure includes a multiplier 111, an adder 112, an accumulator 113, and a bit quantization unit 114. The processing element array 110 can reduce the data with a bit width of (N+M+log2(L)) bits output from the accumulator 113 to a bit width of (X) bits through the bit quantization unit 114. The NPU scheduler 130 can control the bit quantization unit 114 to reduce the bit width of the output data from the least significant bit (LSB) to a predetermined bit of the most significant bit (MSB). When the bit width of the output data is reduced, power consumption, operation amount, and memory usage can be reduced. However, when the bit width is reduced to below a certain length, there may be a problem of a sharp deterioration in the inference accuracy of the artificial neural network model. Therefore, the bit width reduction of the output data, that is, the quantization level, can be determined by comparing the amount of reduction in power consumption, operation amount, and memory usage with the level of reduction in the inference accuracy of the artificial neural network model. The quantization level can be determined by determining the target inference accuracy of the artificial neural network model and testing by gradually reducing the bit width. A quantization level can be determined for each operation value at each layer.

[0168] According to the first processing element PE1 described above, the processing element array 110 can reduce power consumption while increasing the MAC operation speed by adjusting the bit widths of the (N)-bit data and the (M)-bit data of the multiplier 111 and reducing the bit width of the operation value (X) bits by the bit quantization unit 114. In addition, the convolution operation of the artificial neural network can be performed more efficiently.

[0169] The NPU internal memory 120 of the neural processing unit 100 may be a memory system configured in consideration of the MAC operation characteristics and power consumption characteristics of the processing element array 110 .

[0170] For example, the neural processing unit 100 may be configured to reduce the bit width of the operation value of the processing element array 110 in consideration of the MAC operation characteristics and power consumption characteristics of the processing element array 110 .

[0171] The NPU internal memory 120 of the neural processing unit 100 may be configured to minimize the power consumption of the neural processing unit 100 .

[0172] The NPU internal memory 120 of the neural processing unit 100 may be a memory system configured to control memory with low power in consideration of the data size and operation steps of an ongoing artificial neural network model.

[0173] The NPU internal memory 120 of the neural processing unit 100 may be a low-power memory system configured to reuse specific memory addresses in which weight data is stored, taking into account the data size and operation steps of the ongoing artificial neural network model.

[0174] The neural processing unit 100 may provide various activation functions to impart nonlinearity. For example, the neural processing unit 100 may provide a sigmoid function, a hyperbolic tangent function, or a ReLU function. The activation function may be selectively applied after the MAC operation. The operation value of applying the activation function may be referred to as an activation map.

[0175] Figure 3 Picture shows Figure 1 A modified example of the neural processing unit 100.

[0176] Figure 3 The neural processing unit 100 and Figure 1 The processing unit 100 exemplarily shown in FIG. 1 is substantially the same as that in FIG. 1 , except for the processing element array 110 . Therefore, for the convenience of description, redundant description will be omitted.

[0177] Figure 1 The processing element array 110 exemplarily shown in FIG. 1 may include register files RF1 to RF12 corresponding to the processing elements PE1 to PE12 in addition to the plurality of processing elements PE1 to PE12 .

[0178] Figure 3 The plurality of processing elements PE1 to PE12 and the plurality of register files RF1 to RF12 are merely examples for convenience of description, and the number of the plurality of processing elements PE1 to PE12 and the plurality of register files RE1 to RE12 is not limited.

[0179] The size or number of processing element array 110 may be determined by the number of processing elements PE1 to PE12 and register files RF1 to RF12. The size of processing element array 110 and register files RF1 to RF12 may be implemented by an N×M matrix. Here, N and M are integers greater than zero.

[0180] The array size of the processing element array 110 may be designed in consideration of the characteristics of the artificial neural network model in which the neural processing unit 100 operates. As an additional explanation, the memory size of the register file may be determined in consideration of the data size operated by the artificial neural network model, the required operation speed, and the required power consumption.

[0181] The register files RF1 to RF12 of the neural processing unit 100 are static memory units directly connected to the processing elements PE1 to PE12. For example, the register files RF1 to RF12 can be configured by triggers and / or latches. The register files RF1 to RF12 can be configured to store MAC operation values ​​of the corresponding processing elements RF1 to RF12. The register files RF1 to RF12 can be configured to provide weight data and / or node data to the NPU system memory 120 or be configured to be provided with weight data and / or node data from the NPU system memory 120.

[0182] Figure 4 An exemplary artificial neural network (ANN) model is illustrated.

[0183] Hereinafter, the operation of an exemplary artificial neural network model 110a that may be operated in the neural processing unit 100 will be explained.

[0184] Figure 4 The exemplary artificial neural network model 110a may be an artificial neural network trained in the neural processing unit 100 or trained in a separate machine learning device. The artificial neural network model 110a may be an artificial neural network that is trained to perform various reasoning functions, such as object recognition or speech recognition.

[0185] The artificial neural network model 110a may be a deep neural network (DNN).

[0186] However, the artificial neural network model 110a according to the examples of the present disclosure is not limited to deep neural networks.

[0187] For example, the artificial neural network model 110a can be composed of a model such as a fully convolutional network (FCN) having a VGG, VGG16, DenseNET, and an encoder-decoder structure, a deep neural network (DNN) such as SegNet, DeconvNet, DeepLAB, V3+, or U-net, or SqueezeNet, Alexnet, ResNet18, MobileNet-v2, GoogLeNet, Resnet-v2, Resnet50, Resnet101, and Inception-v3, but the present disclosure is not limited thereto. In addition, the artificial neural network model 110a can be an integrated model based on at least two different models.

[0188] The artificial neural network model 110a may be stored in the NPU internal memory 120 of the neural processing unit 100. Alternatively, the artificial neural network model 110a may be implemented to be stored in the memory 200 and then loaded into the neural processing unit 100 during operation of the artificial neural network model 110a.

[0189] In the following, reference will be made to Figure 4 The inference process of an exemplary artificial neural network model 110a performed by the neural processing unit 100 is described.

[0190] The artificial neural network model 110a may be an exemplary deep neural network model, which includes an input layer 110a-1, a first connection network 110a-2, a first hidden layer 110a-3, a second connection network 110a-4, a second hidden layer 110a-5, a third connection network 110a-6, and an output layer 110a-7. However, the present disclosure is not limited to Figure 4 The artificial neural network model shown in . The first hidden layer 110a-3 and the second hidden layer 110a-5 can also be referred to as multiple hidden layers.

[0191] The input layer 110a-1 may exemplarily include input nodes x1 and x2. That is, the input layer 110a-1 may include information about two input values. Figure 1 The NPU scheduler 130 shown in FIG. 3 may be configured to Figure 1 A memory address is set in the NPU internal memory 120 of or 3, and information about the input value from the input layer 110a-1 is stored in the memory address.

[0192] For example, the first connection network 110 a - 2 may include information about six weight values ​​for connecting the nodes of the input layer 110 a - 1 to the nodes of the first hidden layer 110 a - 3 , respectively. Figure 1 The NPU scheduler 130 of the first hidden layer 110a-3 may set a memory address in the NPU internal memory 120, in which information about the weight value of the first connection network 110a-2 is stored. Each weight value is multiplied by the input node value, and the accumulated value of the multiplied value is stored in the first hidden layer 110a-3.

[0193] For example, the first hidden layer 110a-3 may include nodes a1, a2, and a3. That is, the first hidden layer 110a-3 may include information about three node values. Figure 1 The NPU scheduler 130 shown in FIG. 3 sets a memory address in the NPU internal memory 120 for storing information about the node values ​​of the first hidden layer 110 a - 3 .

[0194] For example, the second connection network 110 a - 4 may include information on nine weight values ​​for connecting the nodes of the first hidden layer 110 a - 3 to the nodes of the second hidden layer 110 a - 5 , respectively. Figure 1 The NPU scheduler 130 of the first or second hidden layer 110a-3 may set a memory address for storing information about the weight value of the second connection network 110a-4 in the NPU internal memory 120. The weight value of the second connection network 110a-4 is multiplied by the node value input from the corresponding first hidden layer 110a-3, and the accumulated value of the multiplied value is stored in the second hidden layer 110a-5.

[0195] For example, the second hidden layer 110a-5 may include nodes b1, b2, and b3. That is, the second hidden layer 110a-5 may include information about three node values. The NPU scheduler 130 may set a memory address in the NPU internal memory 120 for storing information about the node values ​​of the second hidden layer 110a-5.

[0196] For example, the third connection network 110a-6 may include information about six weight values ​​that respectively connect the nodes of the second hidden layer 110a-5 and the nodes of the output layer 110a-7. The NPU scheduler 130 may set a memory address for storing information about the weight values ​​of the third connection network 110a-6 in the NPU internal memory 120. The weight values ​​of the third connection network 110a-6 are multiplied by the node values ​​input from the second hidden layer 110a-5, and the accumulated values ​​of the multiplied values ​​are stored in the output layer 110a-7.

[0197] For example, the output layer 110a-7 may include nodes y1 and y2. That is, the output layer 110a-7 may include information about two node values. The NPU scheduler 130 may set a memory address for storing information about the node values ​​of the output layer 110a-7 in the NPU internal memory 120.

[0198] That is, the NPU scheduler 130 may analyze or receive the structure of the artificial neural network model that may be operated in the processing element array 110. The information of the artificial neural network that may be included in the artificial neural network model may include: information about node values ​​of each layer, placement data locality information of the layer or information about the structure, and information about weight values ​​of each connection network connecting nodes of the layer.

[0199] The NPU scheduler 130 is provided with data locality information of the exemplary artificial neural network model 110 a or information about its structure, so that the NPU scheduler 130 can determine the order of operations from input to output of the artificial neural network model 110 a .

[0200] Therefore, considering the scheduling order, the NPU scheduler 130 can set the memory address for storing the MAC operation value of each layer in the NPU internal memory 120. For example, the specific memory address can be the MAC operation value of the input layer 110a-1 and the first connection network 110a-2, or the input data of the first hidden layer 110a-3. However, the present disclosure is not limited to the MAC operation value, and the MAC operation value can also be referred to as an artificial neural network operation value.

[0201] At this time, the NPU scheduler 130 knows that the MAC operation results of the input layer 110a-1 and the first connection network 110a-2 are used as the input of the first hidden layer 110a-3, so that it is controlled to use the same memory address. That is, the NPU scheduler 130 can reuse the MAC operation value based on the data locality information of the artificial neural network model or the information about its structure. Therefore, the memory reusability function of the NPU system memory 120 can be provided.

[0202] That is, the NPU scheduler 130 can store the MAC operation value of the artificial neural network model 110a in a specific area specified in any memory address of the NPU internal memory 120 according to the scheduling order, and use the MAC operation value as input data for the MAC operation in the subsequent scheduling order in the specific area where the MAC operation value is stored.

[0203] MAC operation from the perspective of the first processing element PE1

[0204] The MAC operation will be described in detail from the perspective of the first processing element PE1. The first processing element PE1 may be designated to perform a MAC operation of the node a1 of the first hidden layer 110a-3.

[0205] First, the first processing element PE1 inputs the node x1 data of the input layer 110a-1 to the first input unit of the multiplier 111, and inputs the weight data between the node x1 and the node a1 to the second input unit. The adder 112 adds the operation value of the multiplier 111 and the operation value of the accumulator 113. At this time, when the (L) cycles are 0, there is no accumulated value, so the accumulated value is 0. Therefore, the count value of the operation value adder 112 can be equal to the operation value of the multiplier 111. At this time, the count value of the (L) cycles can be 1.

[0206] Next, the first processing element PE1 inputs the node x2 data of the input layer 110a-1 to the first input unit of the multiplier 111, and inputs the weight data between the node x2 and the node a1 to the second input unit. The adder 112 adds the operation value of the multiplier 111 to the operation value of the accumulator 113. At this time, when (L) cycles is 1, the node x1 data calculated in the previous step and the weight multiplication value between the node x1 and the node a1 are stored. Therefore, the adder 112 generates the MAC operation value of the node x1 and the node x2 corresponding to the node a1.

[0207] Third, the NPU scheduler 130 may complete the MAC operation of the first processing unit PE1 based on the data locality information of the artificial neural network model or the information about its structure. At this time, the initialization reset is input to initialize the accumulator 113. That is, the counter value of (L) cycles may be initialized to 0.

[0208] The bit quantization unit 114 may be appropriately adjusted according to the accumulated value. In other words, as the number of (L) cycles increases, the bit width of the output value increases. At this time, the NPU scheduler 130 may remove the predetermined lower bits so that the bit width of the operation value of the first processing element PE1 is (x) bits.

[0209] MAC operation from the perspective of the second processing element PE2

[0210] The MAC operation will be described in detail from the perspective of the second processing element PE2. The second processing element PE2 may be designated to perform a MAC operation of the node a2 of the first hidden layer 110a-3.

[0211] First, the second processing element PE2 inputs the node x1 data of the input layer 110a-1 to the first input unit of the multiplier 111, and inputs the weight data between the node x1 and the node a2 to the second input unit. The adder 112 adds the operation value of the multiplier 111 and the operation value of the accumulator 113. At this time, when the (L) cycles are 0, there is no accumulated value, so the accumulated value is 0. Therefore, the operation value of the adder 112 can be equal to the operation value of the multiplier 111. At this time, the counter value of the (L) cycles can be 1.

[0212] Secondly, the second processing element PE2 inputs the node x2 data of the input layer 110a-1 to the first input unit of the multiplier 111, and inputs the weight data between the node x2 and the node a2 to the second input unit. The adder 112 adds the operation value of the multiplier 111 to the operation value of the accumulator 113. At this time, when (L) cycles is 1, the node x1 data calculated in the previous step and the weight multiplication value between the node x1 and the node a2 are stored. Therefore, the adder 112 generates the MAC operation value of the node x1 and the node x2 corresponding to the node a2.

[0213] Third, the NPU scheduler 130 may complete the MAC operation of the first processing element PE1 based on the data locality information of the artificial neural network model or the information about its structure. At this time, the initialization reset is input to initialize the accumulator 113. That is, the counter value of (L) cycles may be initialized to 0. The bit quantization unit 114 may be appropriately adjusted according to the accumulated value.

[0214] MAC operation from the perspective of the third processing element PE3

[0215] The MAC operation will be described in detail from the perspective of the third processing element PE3. The third processing element PE3 may be designated to perform a MAC operation of the node a3 of the first hidden layer 110a-3.

[0216] First, the third processing element PE3 inputs the node x1 data of the input layer 110a-1 to the first input unit of the multiplier 111, and inputs the weight data between the node x1 and the node a3 to the second input unit. The adder 112 adds the operation value of the multiplier 111 and the operation value of the accumulator 113. At this time, when the (L) cycles are 0, there is no accumulated value, so the accumulated value is 0. Therefore, the operation value of the adder 112 can be equal to the operation value of the multiplier 111. At this time, the counter value of the (L) cycles can be 1.

[0217] Secondly, the third processing element PE3 inputs the node x2 data of the input layer 110a-1 to the first input unit of the multiplier 111, and inputs the weight data between the node x2 and the node a3 to the second input unit. The adder 112 adds the operation value of the multiplier 111 to the operation value of the accumulator 113. At this time, when (L) cycles is 1, the node x1 data calculated in the previous step and the weight multiplication value between the node x1 and the node a3 are stored. Therefore, the adder 112 generates the MAC operation value of the node x1 and the node x2 corresponding to the node a3.

[0218] Third, the NPU scheduler 130 may complete the MAC operation of the first processing element PE1 based on the data locality information of the artificial neural network model or the information about its structure. At this time, the initialization reset is input to initialize the accumulator 113. That is, the counter value of (L) cycles may be initialized to 0. The bit quantization unit 114 may be appropriately adjusted according to the accumulated value.

[0219] Therefore, the NPU scheduler 130 of the neural processing unit 100 may perform the MAC operation of the first hidden layer 110 a - 3 by simultaneously using the three processing elements PE1 to PE3 .

[0220] MAC operation from the perspective of the fourth processing element PE4

[0221] The MAC operation will be described in detail from the perspective of the fourth processing element PE4. The fourth processing element PE4 may be designated to perform a MAC operation of the node b1 of the second hidden layer 110a-5.

[0222] First, the fourth processing element PE4 inputs the node a1 data of the first hidden layer 110a-3 to the first input unit of the multiplier 111, and inputs the weight data between the node a1 and the node b1 to the second input unit. The adder 112 adds the operation value of the multiplier 111 and the operation value of the accumulator 113. At this time, when the (L) cycles are 0, there is no accumulated value, so the accumulated value is 0. Therefore, the operation value of the adder 112 can be equal to the operation value of the multiplier 111. At this time, the counter value of the (L) cycles can be 1.

[0223] Secondly, the fourth processing element PE4 inputs the node a2 data of the first hidden layer 110a-3 into the first input unit of the multiplier 111, and inputs the weight data between the node a2 and the node b1 into the second input unit. The adder 112 adds the operation value of the multiplier 111 to the operation value of the accumulator 113. At this time, when the (L) cycles are 1, the node a1 data calculated in the previous step and the weight multiplication value between the node a1 and the node b1 are stored. Therefore, the adder 112 generates the MAC operation value of the node a1 and the node a2 corresponding to the node b1. At this time, the counter value of the (L) cycles can be 2.

[0224] Third, the fourth processing element PE4 inputs the node a3 data of the input layer 110a-1 to the first input unit of the multiplier 111, and inputs the weight data between the node a3 and the node b1 to the second input unit. The adder 112 adds the operation value of the multiplier 111 and the operation value of the accumulator 113. At this time, when the (L) cycles are 2, the MAC operation value of the node a1 and the node a2 corresponding to the node b1 calculated in the previous step is stored. Therefore, the adder 112 generates the MAC operation value of the node a1, the node a2, and the node a3 corresponding to the node b1.

[0225] Fourth, the NPU scheduler 130 may complete the MAC operation of the first processing element PE1 based on the data locality information of the artificial neural network model or the information about its structure. At this time, the initialization reset is input to initialize the accumulator 113. That is, the counter value of (L) cycles may be initialized to 0. The bit quantization unit 114 may be appropriately adjusted according to the accumulated value.

[0226] MAC operation from the perspective of the fifth processing element PE5

[0227] The MAC operation will be described in detail from the perspective of the fifth processing element PE5. The fifth processing element PE5 may be designated to perform a MAC operation of the node b2 of the second hidden layer 110a-5.

[0228] First, the fifth processing element PE5 inputs the node a1 data of the first hidden layer 110a-3 to the first input unit of the multiplier 111, and inputs the weight data between the node a1 and the node b2 to the second input unit. The adder 112 adds the operation value of the multiplier 111 and the operation value of the accumulator 113. At this time, when the (L) cycles are 0, there is no accumulated value, so the accumulated value is 0. Therefore, the operation value of the adder 112 can be equal to the operation value of the multiplier 111. At this time, the counter value of the (L) cycles can be 1.

[0229] Secondly, the fifth processing element PE5 inputs the node a2 data of the first hidden layer 110a-3 into the first input unit of the multiplier 111, and inputs the weight data between the node a2 and the node b2 into the second input unit. The adder 112 adds the operation value of the multiplier 111 to the operation value of the accumulator 113. At this time, when the (L) cycles are 1, the node a1 data calculated in the previous step and the weight multiplication value between the node a1 and the node b2 are stored. Therefore, the adder 112 generates the MAC operation value of the node a1 and the node a2 corresponding to the node b2. At this time, the counter value of the (L) cycles can be 2.

[0230] Third, the fifth processing element PE5 inputs the node a3 data of the input layer 110a-1 to the first input unit of the multiplier 111, and inputs the weight data between the node a3 and the node b2 to the second input unit. The adder 112 adds the operation value of the multiplier 111 and the operation value of the accumulator 113. At this time, when the (L) number of cycles is 2, the MAC operation value of the node a1 and the node a2 corresponding to the node b2 calculated in the previous step is stored. Therefore, the adder 112 generates the MAC operation value of the node a1, the node a2, and the node a3 corresponding to the node b2.

[0231] Fourth, the NPU scheduler 130 may complete the MAC operation of the first processing element PE1 based on the data locality information of the artificial neural network model or the information about its structure. At this time, the initialization reset is input to initialize the accumulator 113. That is, the counter value of (L) cycles may be initialized to 0. The bit quantization unit 114 may be appropriately adjusted according to the accumulated value.

[0232] MAC operation from the perspective of the sixth processing element PE6

[0233] The MAC operation will be described in detail from the perspective of the sixth processing element PE6. The sixth processing element PE6 may be designated to perform a MAC operation of the node b3 of the second hidden layer 110a-5.

[0234] First, the sixth processing element PE6 inputs the node a1 data of the first hidden layer 110a-3 to the first input unit of the multiplier 111, and inputs the weight data between the node a1 and the node b3 to the second input unit. The adder 112 adds the operation value of the multiplier 111 and the operation value of the accumulator 113. At this time, when the (L) cycles are 0, there is no accumulated value, so the accumulated value is 0. Therefore, the operation value of the adder 112 can be equal to the operation value of the multiplier 111. At this time, the counter value of the (L) cycles can be 1.

[0235] Secondly, the sixth processing element PE6 inputs the node a2 data of the first hidden layer 110a-3 into the first input unit of the multiplier 111, and inputs the weight data between the node a2 and the node b3 into the second input unit. The adder 112 adds the operation value of the multiplier 111 to the operation value of the accumulator 113. At this time, when the (L) cycles are 1, the node a1 data calculated in the previous step and the weight multiplication value between the node a1 and the node b3 are stored. Therefore, the adder 112 generates the MAC operation value of the node a1 and the node a2 corresponding to the node b3. At this time, the counter value of the (L) cycles can be 2.

[0236] Third, the sixth processing element PE6 inputs the node a3 data of the input layer 110a-1 to the first input unit of the multiplier 111, and inputs the weight data between the node a3 and the node b3 to the second input unit. The adder 112 adds the operation value of the multiplier 111 and the operation value of the accumulator 113. At this time, when the (L) number of cycles is 2, the MAC operation value of the node a1 and the node a2 corresponding to the node b3 calculated in the previous step is stored. Therefore, the adder 112 generates the MAC operation value of the node a1, the node a2, and the node a3 corresponding to the node b3.

[0237] Fourth, the NPU scheduler 130 may complete the MAC operation of the first processing element PE1 based on the data locality information of the artificial neural network model or the information about its structure. At this time, the initialization reset is input to initialize the accumulator 113. That is, the counter value of (L) cycles may be initialized to 0. The bit quantization unit 114 may be appropriately adjusted according to the accumulated value.

[0238] Therefore, the NPU scheduler 130 of the neural processing unit 100 may perform the MAC operation of the second hidden layer 110 a - 5 by simultaneously using the three processing elements PE4 to PE6 .

[0239] MAC operation from the perspective of the seventh processing element PE7

[0240] The MAC operation will be described in detail from the perspective of the seventh processing element PE7. The seventh processing element PE7 may be designated to perform a MAC operation of the node yl of the output layer 110a-7.

[0241] First, the seventh processing element PE7 inputs the node b1 data of the second hidden layer 110a-5 to the first input unit of the multiplier 111, and inputs the weight data between the node b1 and the node y1 to the second input unit. The adder 112 adds the operation value of the multiplier 111 and the operation value of the accumulator 113. At this time, when the (L) cycles are 0, there is no accumulated value, so the accumulated value is 0. Therefore, the operation value of the adder 112 can be equal to the operation value of the multiplier 111. At this time, the counter value of the (L) cycles can be 1.

[0242] Secondly, the seventh processing element PE7 inputs the node b2 data of the second hidden layer 110a-5 into the first input unit of the multiplier 111, and inputs the weight data between the node b2 and the node y1 into the second input unit. The adder 112 adds the operation value of the multiplier 111 to the operation value of the accumulator 113. At this time, when the (L) cycles are 1, the node b1 data calculated in the previous step and the weight multiplication value between the node b1 and the node y1 are stored. Therefore, the adder 112 generates the MAC operation value of the node b1 and the node b2 corresponding to the node y1. At this time, the counter value of the (L) cycles can be 2.

[0243] Third, the seventh processing element PE7 inputs the node b3 data of the input layer 110a-1 to the first input unit of the multiplier 111, and inputs the weight data between the node b3 and the node y1 to the second input unit. The adder 112 adds the operation value of the multiplier 111 and the operation value of the accumulator 113. At this time, when the (L) number of cycles is 2, the MAC operation value of the node b1 and the node b2 corresponding to the node y1 calculated in the previous step is stored. Therefore, the adder 112 generates the MAC operation values ​​of the node b1, the node b2, and the node b3 corresponding to the node y1.

[0244] Fourth, the NPU scheduler 130 may complete the MAC operation of the first processing element PE1 based on the data locality information of the artificial neural network model or the information about its structure. At this time, the initialization reset is input to initialize the accumulator 113. That is, the counter value of (L) cycles may be initialized to 0. The bit quantization unit 114 may be appropriately adjusted according to the accumulated value.

[0245] MAC operation from the perspective of the eighth processing element PE8

[0246] The MAC operation will be described in detail from the perspective of the eighth processing element PE8. The eighth processing element PE8 may be designated to perform a MAC operation of the node y2 of the output layer 110a-7.

[0247] First, the eighth processing element PE8 inputs the node b1 data of the second hidden layer 110a-5 to the first input unit of the multiplier 111, and inputs the weight data between the node b1 and the node y2 to the second input unit. The adder 112 adds the operation value of the multiplier 111 and the operation value of the accumulator 113. At this time, when the (L) cycles are 0, there is no accumulated value, so the accumulated value is 0. Therefore, the operation value of the adder 112 can be equal to the operation value of the multiplier 111. At this time, the counter value of the (L) cycles can be 1.

[0248] Secondly, the eighth processing element PE8 inputs the node b2 data of the second hidden layer 110a-5 into the first input unit of the multiplier 111, and inputs the weight data between the node b2 and the node y2 into the second input unit. The adder 112 adds the operation value of the multiplier 111 to the operation value of the accumulator 113. At this time, when the (L) cycle is 1, the node b1 data calculated in the previous step and the weight multiplication value between the node b1 and the node y2 are stored. Therefore, the adder 112 generates the MAC operation value of the node b1 and the node b2 corresponding to the node y2. At this time, the counter value of the (L) cycle can be 2.

[0249] Third, the eighth processing element PE8 inputs the node b3 data of the input layer 110a-1 to the first input unit of the multiplier 111, and inputs the weight data between the node b3 and the node y2 to the second input unit. The adder 112 adds the operation value of the multiplier 111 and the operation value of the accumulator 113. At this time, when the (L) number of cycles is 2, the MAC operation value of the node b1 and the node b2 corresponding to the node y2 calculated in the previous step is stored. Therefore, the adder 112 generates the MAC operation values ​​of the node b1, the node b2, and the node b3 corresponding to the node y2.

[0250] Fourth, the NPU scheduler 130 may complete the MAC operation of the first processing element PE1 based on the data locality information of the artificial neural network model or the information about its structure. At this time, the initialization reset is input to initialize the accumulator 113. That is, the counter value of (L) cycles may be initialized to 0. The bit quantization unit 114 may be appropriately adjusted according to the accumulated value.

[0251] Therefore, the NPU scheduler 130 of the neural processing unit 100 can perform the MAC operation of the output layer 110a-7 by using two processing elements PE7 and PE8 at the same time.

[0252] When the MAC operation of the eighth processing element PE8 is completed, the reasoning operation of the artificial neural network model 110a can be completed. That is, the artificial neural network model 110a can determine that the reasoning operation of one frame is completed. If the neural processing unit 100 infers motion image data in real time, the image data of subsequent frames can be input to the input nodes x1 and x2 of the input layer 110a-1. At this time, the NPU scheduler 130 can store the image data of subsequent frames in the memory address that stores the input data of the input layer 110a-1. When this process is repeated at each frame, the neural processing unit 100 can process the reasoning operation in real time. In addition, the memory address that has been set can be reused.

[0253] according to Figure 4In summary of the artificial neural network model 110a, the NPU scheduler 130 of the neural processing unit 100 may determine an operation scheduling order for the inference operation of the artificial neural network model 110a based on the data locality information of the artificial neural network model 110a or the information about its structure. The NPU scheduler 130 may set a memory address required by the NPU internal memory 120 based on the operation scheduling order. The NPU scheduler 130 may set a memory address of a reused memory based on the data locality information of the artificial neural network model 110a or the information about its structure. The NPU scheduler 130 specifies the processing elements PE1 to PE8 required for the inference operation to perform the inference operation.

[0254] In other words, when the number of weight data connected to one node increases to L, the number of (L) cycles of the accumulator of the processing element can be set to L-1. That is, even if the weight data of the artificial neural network increases, the accumulator increases the number of times the accumulator is accumulated to easily perform the inference operation.

[0255] That is, the NPU scheduler 130 of the neural processing unit 100 according to the example of the present disclosure can control the processing element array 110 and the NPU internal memory 120 based on the data locality information of the artificial neural network model and the information about its structure (including the data locality information of the input layer 110a-1, the first connection network 110a-2, the first hidden layer 110a-3, the second connection layer 110a-4, the second hidden layer 110a-5, the third connection layer 110a-6 and the output layer 110a-7 and the information about its structure).

[0256] That is, the NPU scheduler 130 may set the memory address values ​​corresponding to the node data of the input layer 110a-1, the weight data of the first connection network 110a-2, the node data of the first hidden layer 110a-3, the weight data of the second connection layer 110a-4, the node data of the second hidden layer 110a-5, the weight data of the third connection layer 110a-6, and the node data of the output layer 110a-7 in the NPU storage system 110.

[0257] Hereinafter, the scheduling of the NPU scheduler 130 will be described in detail. The NPU scheduler 130 may schedule the operation order of the artificial neural network model based on data locality information of the artificial neural network model or information about its structure.

[0258] The NPU scheduler 130 may acquire a memory address value in which node data of a layer of the artificial neural network model and weight data of a connection network are stored based on data locality information of the artificial neural network model or information about its structure.

[0259] For example, the NPU scheduler 130 may obtain the memory address values ​​of the node data of the layers of the artificial neural network model stored in the main memory and the weight data of the connection network. Therefore, the NPU scheduler 130 may obtain the node data of the layers of the artificial neural network model to be driven and the weight data of the connection network from the main memory to store the data in the NPU internal memory 120. The node data of each layer may have a corresponding memory address value. The weight data of each connection network may have a corresponding memory address value.

[0260] The NPU scheduler 130 may schedule the operation order of the processing element array 110 based on data locality information of the artificial neural network model or information about its structure (eg, placement data locality information of an artificial neural network layer of the artificial neural network model or information about its structure).

[0261] For example, the NPU scheduler 130 may obtain weight data having four artificial neural network layers and weight values ​​of three layers connecting these layers, that is, connection network data. In this case, a method for scheduling a processing order by the NPU scheduler 130 based on data locality information of an artificial neural network model or information about its structure will be described below by way of example.

[0262] For example, the NPU scheduler 130 may set the input data for the inference operation to the node data of the first layer of the input layer 110a-1 of the artificial neural network model 110a and schedule to first perform a multiplication and accumulation (MAC) operation on the node data of the first layer and the weight data of the first connection network corresponding to the first layer. In the following, for the convenience of description, the corresponding operation is referred to as the first operation, the result of the first operation is referred to as the first operation value, and the corresponding schedule is referred to as the first schedule.

[0263] For example, the NPU scheduler 130 may set the first operation value to the node data of the second layer corresponding to the first connection network after the first scheduling, and schedule to perform a MAC operation of the node data of the second layer and the weight data of the second connection network corresponding to the second layer. Hereinafter, for the convenience of description, the corresponding operation is referred to as the second operation, the result of the second operation is referred to as the second operation value, and the corresponding scheduling may be referred to as the second scheduling.

[0264] For example, the NPU scheduler 130 may set the second operation value to the node data of the third layer corresponding to the second connection network during the second scheduling, and schedule the execution of the MAC operation of the node data of the third layer and the weight data of the third connection network corresponding to the third layer. Hereinafter, for the convenience of description, the corresponding operation is referred to as the third operation, the result of the third operation is referred to as the third operation value, and the corresponding scheduling is referred to as the third scheduling.

[0265] For example, the NPU scheduler 130 may set the third operation value to the node data of the fourth layer (the fourth layer is the output layer 110a-7 corresponding to the third connection network), and schedule to store the inference result stored in the node data of the fourth data in the NPU internal memory 120. Hereinafter, for convenience of description, the corresponding schedule may be referred to as the fourth schedule. The inference result value is sent to each component to be utilized.

[0266] For example, when the inference result value is a result value of detecting a specific keyword, the neural processing unit 100 sends the inference result to the central processing unit to perform an operation corresponding to the specific keyword.

[0267] For example, the NPU scheduler 130 may drive the first to third processing elements PE1 to PE3 in a first schedule.

[0268] For example, the NPU scheduler 130 may drive the fourth to sixth processing elements PE4 to PE6 in the second schedule.

[0269] For example, the NPU scheduler 130 may drive the seventh and eighth processing elements PE7 and PE8 in the third schedule.

[0270] For example, the NPU scheduler 130 may output the inference result in the fourth schedule.

[0271] In summary, the NPU scheduler 130 can control the NPU internal memory 120 and the processing element array 110 to perform operations in the order of the first schedule, the second schedule, the third schedule, and the fourth schedule. That is, the NPU scheduler 130 can be configured to control the NPU internal memory 120 and the processing element array 110 to perform operations according to the set scheduling order.

[0272] In summary, the neural processing unit 100 according to the example of the present disclosure can be configured to schedule the processing order based on the structure of the layer of the artificial neural network and the operation order data corresponding to the structure. The processing order to be scheduled can be at least one. For example, the neural processing unit 100 can predict all operation orders so that subsequent operations can be scheduled, or operations can be scheduled in a specific order.

[0273] The NPU scheduler 130 may improve memory reuse by controlling the NPU internal memory 120 by utilizing a scheduling order based on data locality information of the artificial neural network model or information about its structure.

[0274] According to the characteristics of the artificial neural network operation driven in the neural processing unit 100 according to the examples of the present disclosure, the operation value of one layer can be used as input data of a subsequent layer.

[0275] Therefore, when the neural processing unit 100 controls the NPU internal memory 120 according to the scheduling order, the memory reuse rate of the NPU internal memory 120 can be improved.

[0276] Specifically, when the NPU scheduler 130 is configured to be provided with data locality information of the artificial neural network model or information about its structure, and the order of executing the operations of the artificial neural network is calculated by the provided data locality information of the artificial neural network model or information about its structure, the NPU scheduler 130 can identify the operation results of the node data of a specific layer of the artificial neural network model and the weight data of a specific connection network as the corresponding node data of the subsequent layer. Therefore, the NPU scheduler 130 can reuse the value of the memory address storing the corresponding operation result for subsequent operations.

[0277] For example, the first operation value of the first schedule is set as the second-layer node data of the second schedule. More specifically, the NPU scheduler 130 can reset the memory address value corresponding to the first operation value of the first schedule stored in the NPU internal memory 120 to the memory address value corresponding to the node data of the second layer of the second schedule. That is, the memory address value can be reused. Therefore, the NPU scheduler 130 reuses the memory address value of the first schedule, so that the NPU internal memory 120 can use the data as the second-layer node data of the second schedule without a separate memory write operation.

[0278] For example, the second operation value of the second schedule is set as the third-layer node data of the third schedule. More specifically, the NPU scheduler 130 can reset the memory address value corresponding to the second operation value of the second schedule stored in the NPU internal memory 120 to the memory address value corresponding to the node data of the third layer of the third schedule. In other words, the memory address value can be reused. Therefore, the NPU scheduler 130 reuses the memory address value of the second schedule, so that the NPU internal memory 120 can use the data as the third-layer node data of the third schedule without a separate memory write operation.

[0279] For example, the third operation value of the third schedule is set as the fourth-layer node data of the fourth schedule. More specifically, the NPU scheduler 130 can reset the memory address value corresponding to the third operation value of the third schedule stored in the NPU internal memory 120 to the memory address value corresponding to the node data of the fourth layer of the fourth schedule. That is, the memory address value can be reused. Therefore, the NPU scheduler 130 reuses the memory address value of the third schedule, so that the NPU internal memory 120 can use the data as the fourth-layer node data of the fourth schedule without a separate memory write operation.

[0280] In addition, the NPU scheduler 130 may be configured to determine whether to reuse the scheduling order and memory to control the NPU internal memory 120. In this case, the NPU scheduler 130 analyzes the data locality information of the artificial neural network model or the information about its structure to provide optimized scheduling. In addition, data required for operations that can reuse the memory is not repeatedly stored in the NPU internal memory 120, so that the memory usage can be reduced. In addition, the NPU scheduler 130 can optimize the NPU internal memory 120 by calculating to reduce the memory usage as much as the memory is reused.

[0281] The neural processing unit 100 according to the example of the present disclosure may be configured such that a variable value is input to an (N)-bit input as a first input of the first processing element PE1, and a constant value is input to an (M)-bit input as a second input. In addition, the configuration may be set in the same manner in other processing elements of the processing element array 110. That is, one input of the processing element may be configured to receive a variable value, and another input may be configured to receive a constant value. Therefore, the number of times the data of the constant value is updated may be reduced.

[0282] At this time, the NPU scheduler 130 uses the data locality information of the artificial neural network model 110a or the information about its structure to set the input layer 110a-1, the first hidden layer 110a-3, the second hidden layer 110a-5 and the output layer 110a-7 as variables, and sets the weight data of the first connection network 110a-2, the weight data of the second connection network 110a-4 and the weight data of the third connection network 110a-6 as constants. That is, the NPU scheduler 130 can distinguish between constant values ​​and variable values. However, the present disclosure is not limited to constant and variable data types, but distinguishes frequently changing values ​​from unchanged values ​​to improve the reuse rate of the NPU internal memory 120.

[0283] That is, the NPU system memory 120 may be configured to save the weight data of the connection network stored in the NPU system memory 120 while maintaining the inference operation of the neural processing unit 100. Therefore, the memory read and write operations may be reduced.

[0284] That is, the NPU system memory 120 may be configured to reuse MAC operation values ​​stored in the NPU system memory 120 while maintaining the inference operation.

[0285] That is, the number of updates of the data of the memory address storing the input data (N) bits of the first input unit of the processing element of the processing element array 110 may be greater than the number of updates of the data of the memory address storing the input data (M) bits of the second input unit. That is, the number of updates of the data of the second input unit may be less than the number of updates of the data of the first input unit.

[0286] Hereinafter, a convolutional neural network (CNN) which is a type of deep neural network (DNN) in an artificial neural network will be mainly described.

[0287] A convolutional neural network can be a combination of one or more convolutional layers, pooling layers, and fully connected layers. CNN has a structure suitable for two-dimensional data learning and reasoning, and can be trained by the back-propagation algorithm.

[0288] Figure 5A The figure shows the basic structure of a convolutional neural network.

[0289] refer to Figure 5A , the input image may be represented by a two-dimensional matrix consisting of rows having a specific size and columns having a specific size. The input image may have a plurality of channels and the channel may indicate the number of color components of the input data image.

[0290] The convolution process refers to performing a convolution operation with a kernel while accessing the input image at specified intervals.

[0291] When the convolutional neural network moves from the current layer to the next layer, the weight values ​​between the layers are reflected by the convolution to pass the weight values ​​to the next layer.

[0292] For example, a convolution is defined by two main parameters, the size of the input image (usually 1×1, 3×3, and 5×5 matrices) and the depth (number of kernels) of the output feature map that can be computed by the convolution. A convolution can start at depth 32, continue to depth 64, and end at depth 128 or 256.

[0293] Convolution can operate by sliding a window with size 3×3 or 5×5 over the 3D input feature map, stopping at all locations and extracting 3D blocks of neighboring features.

[0294] The 3D block can be converted to a 1D vector through a tensor product with the same learned weight matrix called weights. This vector can be reassembled spatially into a 3D output map. All spatial locations of the output feature map can correspond to the same location of the input feature map.

[0295] A convolutional neural network can include a convolutional layer that performs a convolution operation between a kernel (that is, a weight matrix) trained through multiple gradient update iterations during the learning process and the input data. If (m,n) is set as the kernel size and W is set as the weight value, the convolutional layer calculates the inner product to perform the convolution of the input data and the weight matrix.

[0296] The step size that the kernel slides over the input data is called the stride, and the kernel area (m×n) can be called the receptive field. The same convolution kernel is applied to different positions of the input, which may reduce the number of kernels to learn. This also makes position-invariant learning possible, where if there is an important pattern in the input, the convolution filter can learn that pattern regardless of the position in the sequence.

[0297] A convolutional neural network can be adjusted or trained so that input data is connected to a specific output estimate. The convolutional neural network can be adjusted using backpropagation based on comparisons between the output estimate and the ground truth until the output estimate gradually matches or approaches the ground truth.

[0298] Convolutional neural networks can be trained by adjusting the weights between neurons based on the difference between true data (ground through) and the actual output.

[0299] Figure 5B Diagram showing the overall operation of a convolutional neural network.

[0300] refer to Figure 5B , the input image is a two-dimensional matrix of size 5×5. In addition, Figure 5B In this example, three nodes are used, namely channel 1, channel 2, and channel 3.

[0301] First, the convolution operation of layer 1 will be described.

[0302] The input image is convolved with kernel 1 of channel 1 at the first node of layer 1, and the resulting output feature is Figure 1 In addition, the input image is convolved with kernel 2 of channel 2 at the second node of layer 1, and the resulting output feature is Figure 2 The input image is convolved with kernel 3 of channel 3 at the third node, and the resulting output feature is Figure 3 .

[0303] Next, the pooling operation of layer 2 will be described.

[0304] Features output from layer 1 Figure 1 ,feature Figure 2 and Features Figure 3is input to the three nodes of layer 2. Layer 2 receives the feature map output from layer 1 as input to perform pooling. Pooling can reduce the size in the matrix or emphasize specific values ​​in the matrix. Pooling methods can include maximum pooling, average pooling, and minimum pooling. Maximum pooling is used to collect the maximum value within a specific area of ​​the matrix, and average pooling is used to calculate the average value within a specific area.

[0305] exist Figure 5B In the example, the feature map of 5×5 matrix is ​​reduced to 4×4 matrix through pooling.

[0306] Specifically, the first node of layer 2 takes the feature of channel 1 Figure 1 Pooling is performed as input and then outputs a 4×4 matrix. The second node of layer 2 takes the features of channel 2 Figure 2 The third node of layer 2 takes the features of channel 3 as input and then outputs a 4×4 matrix. Figure 3 It takes as input the pooling and outputs a 4×4 matrix.

[0307] Next, the convolution operation of layer 3 will be described.

[0308] The first node of layer 3 receives the output from the first node of layer 2 as input to perform convolution with kernel 4 and outputs the result thereof. The second node of layer 3 receives the output from the second node of layer 2 as input to perform convolution with kernel 5 of channel 2 and outputs the result thereof. Similarly, the third node of layer 3 receives the output from the third node of layer 2 as input to perform convolution with kernel 6 of channel 3 and outputs the result thereof.

[0309] As mentioned above, convolution and pooling are repeated, and finally, as Figure 5A As shown, the fully connected layer can be output. The output can be input into the artificial neural network again for image recognition.

[0310] Hereinafter, SoC will be mainly explained, but the disclosure of this specification is not limited to SoC, and the content of this disclosure is also applicable to system-level package (SIP) or printed circuit board (PCB) substrate-level system. For example, each functional component is implemented by an independent semiconductor chip and connected by a system bus, and the system bus is implemented by a conductive pattern formed on the PCB.

[0311] Fig. 6A Shown include Figure 1 An exemplary architecture of a system on a chip (SoC) with an NPU of EM6400 or EM6400.

[0312] refer to Fig. 6A , the exemplary SoC 100 includes a plurality of functional components, a system bus 500 , an in-system component tester (ICT) 600 , and a plurality of test wrappers 700 a , 700 b , . . . , and 700 g , collectively referred to as test wrappers 700 .

[0313] The wrapper can adapt the interface of the components that communicate with each other. Therefore, the wrapper can also be called an interface circuit.

[0314] The multiple functional components may include an NPU core array 100-1, a central processing unit (CPU) core array 200, a graphics processing unit (GPU) core array 300, an internal memory 400, a memory controller 450, an input / output (I / O) interface 800 and a field programmable gate array (FPGA) 900.

[0315] Examples of the present disclosure are not limited thereto, and at least some of the plurality of functional components may be removed. Examples of the present disclosure are not limited thereto, and may further include other functional components in addition to the plurality of functional components described above.

[0316] NPU, CPU and GPU are collectively referred to as universal processing unit (UPI), application processing unit (APU) or application-specific processing unit (ADPU).

[0317] Each NPU core in array 110-1 may refer to Figure 1 or NPU100 in 3. In other words, Figure 1 An array 110 - a of multiple NPUs 100 of or 3 is illustrated in FIG6 .

[0318] Similarly, array 200 may include multiple CPU cores. Array 300 may include multiple GPU cores.

[0319] The array 100-1 of NPU cores may be connected to the system bus 500 via a wrapper 700a. Similarly, the array 200 of CPU cores may be connected to the system bus 500 via a wrapper 700b. Similarly, the array 300 of GPU cores may be connected to the system bus 500 via a wrapper 700c.

[0320] The internal memory 400 may be connected to the system bus 500 via the wrapper 700d. The internal memory 400 may be shared by the CPU core, the GPU core, and the NPU core.

[0321] The memory controller 400 connected to the external memory may be connected to the system bus 500 via the wrapper 700 e.

[0322] The system bus 500 may be implemented by a conductive pattern formed on a semiconductor die. The system bus enables high-speed communication. For example, the CPU core, GPU core, and NPU core may read data from or write data to the internal memory 400 through the system bus 500. Further, the CPU core, GPU core, and NPU core may read data from or write data to an external memory through a memory controller 450.

[0323] The ICT 600 may be connected to the system bus 500 through a dedicated signaling channel. In addition, the ICT 600 may be connected to a plurality of wrappers 700 through a dedicated signaling channel.

[0324] Each wrapper 700 may be connected to the ICT 600 through a dedicated signaling channel. In addition, each wrapper 700 may be connected to the system bus 500 through a dedicated signaling channel. In addition, each wrapper 700 may be connected to a corresponding functional component in the SoC through a dedicated signaling channel.

[0325] To this end, each wrapper 700 may be designed to be located between a corresponding functional component in the SoC and the system bus 500 .

[0326] For example, the first wrapper 700a may be connected to the array 100-1 of NPU cores, the system bus 500, and the ICT 600 through a dedicated signaling channel. The second wrapper 700a may be connected to the CPU core array 200, the system bus 500, and the ICT 600 through a dedicated signaling channel. The wrapper 700c may be connected to the GPU core array 300, the system bus 500, and the ICT 600 through a dedicated signaling channel. The fourth wrapper 700d may be connected to the internal memory 200, the system bus 500, and the ICT 600 through a dedicated signaling channel. The fifth wrapper 700e may be connected to the memory controller 450, the system bus 500, and the ICT 600 through a dedicated signaling channel. The sixth wrapper 700f may be connected to the I / O interface 800, the system bus 500, and the ICT 600 through a dedicated signaling channel. The seventh wrapper 700g may be further connected to the I / O interface 800. It may be configured to communicate with the outside through the seventh wrapper 700g.

[0327] The ICT 600 may monitor the system bus 500 directly or monitor the status of a plurality of functional components through each wrapper 700. Each functional component may be in an idle state or a busy state.

[0328] When a functional component in an idle state is found, the ICT 600 may select the corresponding functional component as a component under test (CUT).

[0329] If multiple functional components are in an idle state, the ICT 600 may select any one functional component as the CUT according to a predetermined rule.

[0330] If a plurality of functional components are in an idle state, the ICT 600 may randomly select any one functional component as a CUT. By doing so, the ICT 600 may cut off the connection between the functional component selected as the CUT and the system bus 500 or isolate it from the system bus 500. To this end, the ICT 600 may instruct the wrapper 700 connected to the functional component selected as the CUT to be cut off or isolated. More specifically, the ICT 600 cuts off the connection between the functional component selected as the CUT and the system bus 500 through the wrapper 700, and then may instruct the wrapper 700 to send a signal to the system bus 500 instead of the functional component selected as the CUT. At this time, the signal transmitted to the system bus 500 may be a signal transmitted to the system bus 500 when the functional component selected as the CUT is in an idle state. To this end, when the functional component selected as the CUT is in an idle state, the wrapper 700 may monitor (or eavesdrop) and store the signal transmitted to the system bus 500. The corresponding wrapper 700 regenerates the stored signal to transmit the regenerated signal to the system bus 500. At the same time, the corresponding wrapper 700 can detect the signal from the system bus 500.

[0331] Thereafter, the ICT 600 may test the functional component selected as the CUT.

[0332] Specifically, the rules may include priority rules based on tasks to be performed, priority rules between functional components, rules based on the presence or absence of spare components for corresponding functional components, rules defined by the number of tests, and rules defined by previous test results.

[0333] For example, when the priority rule of the task indicates that the operation of the GPU has a higher priority than the operation of the CPU, the GPU can be tested preferentially between the CPU and the GPU in an idle state. When the priority rule between functional components indicates that the priority of the CPU is higher than the priority of the GPU, the CPU can be tested preferentially between the CPU and the GPU in an idle state. According to the rule of the presence or absence of spare parts, when the GPU has three cores and the CPU has six cores, the GPU with fewer cores can be tested preferentially. According to the rule defined by the number of tests, when the CPU is tested 3 times and the GPU is tested 5 times, the tested CPU can be tested preferentially. According to the rule defined by the last test result, when the last test result of the CPU is found to be abnormal and the last test result of the GPU is normal, the CPU can be tested preferentially.

[0334] When a conflict occurs due to accessing a functional component selected as a CUT from the system bus 500 at the start of a test or during a test, the ICT 600 can detect the conflict.

[0335] If so, the ICT 600 may stop (abort) the test and drive a backoff timer on the conflict.

[0336] The ICT 600 may restore the connection between the functional component selected as the CUT and the system bus 500 .

[0337] Meanwhile, when the backoff time of the backoff timer for the conflict expires, the ICT 600 may monitor whether the functional component enters the idle state again. If the functional component enters the idle state again, the ICT 600 may select the functional component as the CUT again.

[0338] If no conflicts are detected, the ICT 600 may continue testing and, when the testing is complete, analyze the test results.

[0339] Testing can be used to verify whether a component of the system has a defect in its manufacture, has been damaged, or has been broken. The damage or breaking may be caused by fatigue stress or physical stress (such as heat or electromagnetic pulse (EMP)) caused by repeated use. That is, it can be configured to detect defects based on the test results.

[0340] The test performed on the NPU will be described. As described below, there are two types of tests, including functional tests and scan tests.

[0341] First, when performing a functional test on the NPU, ICT600 can input a predetermined ANN test model and test input to the NPU. When the NPU uses the input ANN test model to output the inference result of the test input, ICT 600 compares the expected inference result with the inference result from the NPU to analyze whether the NPU is normal or defective. For example, when the ANN test model is a predetermined CNN and the test input is a simple test image, the NPU uses the ANN test model to perform convolution and pooling on the test image and output a fully connected layer.

[0342] Next, when performing a scan test on the NPU, as will be described below, the ICT 600 can thread the flip-flops in the NPU through the scan chain. The ICT 600 can import the test input to at least one flip-flop and obtain the test result from the operation of the combinational logic of the flip-flop to analyze whether the NPU is defective or normal at runtime.

[0343] The test performed by ICT 600 may be a test performed before the mass-produced SoC comes out of the factory to determine the fair quality. According to the present disclosure, it should be noted that the test for determining the fair quality can also be performed during the runtime of the SoC. That is, according to the known technology, it is only possible to perform a test to determine the fairness of the quality before the SoC leaves the factory. However, according to the present disclosure, functional components in an idle state are found from multiple functional components in the SoC for sequential testing, so that the SoC can be tested for fair quality at runtime.

[0344] As a result of the test analysis, when it is determined that the corresponding functional component is normal, the ICT 600 returns the connection with the functional component to the system bus 500. Specifically, the ICT 600 may disconnect the connection between the wrapper 700 and the system bus 500, and restore the connection between the functional component and the system bus 500. More specifically, the ICT 600 may initialize the functional component to be connected to the system bus 500, and then instruct the wrapper 700 to stop the signal to be transmitted to the system bus 500.

[0345] However, if the test analysis results are determined to be defective, the ICT 600 may repeat the test multiple times.

[0346] When the functional component is determined to be defective as a result of multiple repeated tests, that is, when it is determined that the functional component in the SoC is defective, has been damaged, or has been damaged during its manufacturing process, the ICT 600 can deactivate the functional component.

[0347] Alternatively, when an error code included in the one-time test analysis result indicates that a functional component in the SoC is defective, damaged, or broken during its manufacturing process, the ICT 600 may disable the functional component.

[0348] In order to disable the functional component, the ICT 600 may cut off or disconnect the connection of the functional component determined to be defective to isolate the functional component determined to be defective from the system bus 500. Alternatively, in order to disable the defective functional component, the ICT 600 may power off (turn off) the functional component. When the functional component is powered off, the defective functional component can be prevented from malfunctioning, and the power consumption of the SoC can be reduced.

[0349] In addition, to disable the defective functional component, the ICT 600 may revoke the address of the functional component on the system bus 500 or send a signal for deletion to the system bus 500. That is, the ICT 600 may send a signal for deleting the address of the defective functional component to the component having the address used on the system bus 500.

[0350] At the same time, when deactivation is complete, the ICT 600 can determine whether there is a spare component for the functional component. Even if the spare component may exist, the ICT 600 can activate the spare component when the spare component is not in an active state. That is, the ICT 600 can send a signal including a request for updating the address of the activated spare component in the table to the component having the address table used on the system bus 500.

[0351] When an address on the system bus 500 is not allocated to the spare component in the disabled state, the ICT 600 may send a signal to the system bus 500 for reallocating the address of the defective functional component to the spare component.

[0352] After monitoring whether the spare component is in an idle state, the ICT 600 may perform a test.

[0353] When there is no space for the disabled functional component, the ICT 600 can allow the FPGA 900 to be programmed to mimic the same operation as the disabled functional component. Information used to program the FPGA 900 can be stored in the internal memory 400. Alternatively, information used to program the FPGA 900 can be stored in a cache memory of the FPGA 900.

[0354] As described above, when FPGA 900 is programmed to emulate the same operation as a disabled functional component, ICT 600 may send a signal including a request to update an address table used in system bus 500. Alternatively, a signal including a request to reallocate the address of the failed functional component to the FPGA may be sent to system bus 500. In other words, the existing address of the FPGA may be revoked and replaced with the address of the failed functional component.

[0355] On the other hand, Fig. 6A Unlike the example shown, FPGA 900 may not be directly connected to system bus 500, but may be connected to system bus 500 through a separate wrapper. In other words, a separate wrapper may be added between system bus 500 and FPGA 900.

[0356] When at least one functional component is determined to be defective, the SoC may be configured to display a warning message on a display device that is communicable with the SoC.

[0357] When at least one functional component is determined to be defective, the SoC can be configured to send a warning message to a server that can communicate with the SoC. Here, the server can be a server of the manufacturer or a server of a service center. As described above, according to the present disclosure, the ICT and the wrapper are combined in the SoC, and the test can be performed during the runtime of the SoC.

[0358] Figure 6B It is shown Fig. 6A FIG. 1 is a view of a first modified example of FIG.

[0359] In the following, only Fig. 6A Different contents will be described below, and substantially the same contents will refer to the above contents.

[0360] First, with Fig. 6A Different from the Figure 6B In the embodiment, the wrappers 700, namely 700a, 700b, 700c, 700d, 700e, 700f and 700g, can be arranged to be located close to the system bus 500. In this case, there is an advantage that the SoC design and manufacture will be much simpler.

[0361] As a specific example, Figure 6B As shown, the wrapper 700 can be arranged to surround the system bus 500. When the wrapper 700 is arranged to surround the system bus 500 in this manner, the system bus 500 can be safely isolated. Specifically, since the system bus 500 can be isolated by using Figure 6B The architecture shown is used to safely isolate the functional components, thereby preventing erroneous information / instructions from being transmitted to any functional component via the system bus 500. Figure 6B The illustrated architecture can safely isolate the system bus 500 so that even if any one functional component fails, the impact on other functional components can be prevented.

[0362] On the other hand, unlike Fig. 6A , the FPGA 900 may not be directly connected to the system bus 500, but may be connected to the system bus 500 through the wrapper 700h. That is, the wrapper 700h may be added between the system bus 500 and the FPGA 900.

[0363] Figure 6C It is shown Fig. 6A An example diagram of a second modified example.

[0364] In the following, only Fig. 6A Or 6B will be described with different contents, and basically the same contents will refer to the above contents.

[0365] First, unlike that shown in 6A or 6B, Figure 6CIn the embodiment of the present invention, each wrapper can be arranged to surround each functional component. In other words, each functional component can be implemented in the form of an independent island in the wrapper. In addition, the channel that connects each functional component to the system bus is implemented in the form of a gate bridge, and the channel in the form of a gate bridge can be opened and closed by each wrapper under the control of the ICT 600. Specifically, in the first wrapper 700a, the NPU 100-1 can be implemented in the form of an island. Similarly, the CPU 200 can be implemented in the form of an island in the second wrapper 700b. The GPU 300 can be implemented in the form of an island in the third wrapper 700c. The internal memory 400 can be implemented in the form of an island in the fourth wrapper 700d. The memory controller 450 can be implemented in the form of an island in the fifth wrapper 700e. In the sixth wrapper 700f, the I / O interface 800 can be implemented in the form of an island. The FPGA 900 can be implemented in the form of an island in the seventh wrapper 700h.

[0366] Fig.6D The description includes Figure 1 or Figure 3 An example diagram of an exemplary architecture of a computing system of an NPU is shown in FIG.

[0367] In the following, only Fig. 6A Different contents will be described below, and substantially the same contents will refer to the above contents.

[0368] First, with Fig. 6A Different from the Fig.6D In the embodiment, a plurality of functional components, a system bus 500, an ICT 600, and a wrapper 700 may be mounted on a board substrate 2000 instead of a SoC. The board substrate may be a printed circuit board (PCB). Alternatively, Fig.6D The implementation shown may be a system in a package (SiP).

[0369] The plurality of functional components may include an NPU 100 - 1 , a central processing unit (CPU) 200 , a graphics processing unit (GPU) 300 , an internal memory 400 , a memory controller 450 , a second memory 470 , an input output (I / O) interface 800 , and a field programmable gate array (FPGA) 900 .

[0370] The first memory 400 may be implemented in various forms such as dynamic RAM (DRAM), synchronous dynamic random access memory (SDRAM), static RAM (SRAM), magnetic random access memory (MRAM), resistive random access memory (RRAM), and high bandwidth memory (HBM). However, the present disclosure is not limited thereto.

[0371] The first memory 400 may include a memory controller. Fig.6D , the memory controller 450 for the second memory 470 is shown as existing independently.

[0372] Similarly, the second memory 470 may be implemented in various forms such as dynamic RAM (DRAM), synchronous dynamic random access memory (SDRAM), static RAM (SRAM), magnetic random access memory (MRAM), resistive random access memory (RRAM), and high bandwidth memory (HBM). However, the present disclosure is not limited thereto.

[0373] The first memory 400 or the second memory 470 may be configured to store at least one of the inference data, weight data, and feature map data of the artificial neural network model processed by the NPU 100-1. The inference data may be an input signal of the artificial neural network model.

[0374] Fig. 6E It is shown Fig.6D An example diagram of a first modified example.

[0375] Hereinafter, only the contents different from the above contents will be described, and the contents which are substantially the same will refer to the above contents.

[0376] First, with Fig.6D Different from the Fig. 6E In the embodiment, the wrappers 700, namely 700a, 700b, 700c, 700d, 700e, 700f and 700g, may be arranged to be located close to the system bus 500. In this case, advantageously, the design and manufacturing may be simpler.

[0377] As a specific example, Fig. 6E As shown, the wrapper 700 can be arranged around the system bus 500. When the wrapper 700 is arranged around the system bus 500 in this way, the system bus 500 can be safely isolated. Specifically, since the system bus 500 can be isolated by using Fig. 6E The architecture shown is used to safely isolate, thereby inhibiting incorrect information / instructions from being transmitted to any functional component via the system bus 500. In addition, since the system bus 500 can be used Fig. 6E The architecture shown is used to safely isolate components so that even if any one functional component fails, the impact on other functional components can be prevented.

[0378] On the other hand, unlike Fig.6D, the FPGA 900 is not directly connected to the system bus 500, but is connected to the system bus 500 through the wrapper 700h. That is, the wrapper 700h can be added between the system bus 500 and the FPGA 900.

[0379] Fig. 6F It is shown Fig.6D An example diagram of a second modified example.

[0380] In the following, only Fig.6D Or 6E will be described with different contents, and basically the same contents will refer to the above contents.

[0381] First, with Fig.6D or 6E, in Fig. 6F In the embodiment, each wrapper can be arranged to surround each functional component. In other words, each functional component can be implemented in the form of an independent island within the wrapper. In addition, the channel connecting each functional component to the system bus is implemented in the form of a gate bridge, and the channel in the form of a gate bridge can be opened and closed by each wrapper under the control of the ICT 600.

[0382] In the following, in order to have a deeper understanding of the above content, a more detailed explanation will be given in conjunction with the directory.

[0383] I. Why is runtime testing important?

[0384] Various studies have been conducted to prevent potential accidents that may be caused by hardware defects in autonomous computing systems.

[0385] Among the various tests, pre-deployment testing is included. According to this testing technology, all hardware designs are checked before the product is sold to customers. After manufacturing is completed, the design is tested from various angles to detect and correct various problems that may be found during actual operation. For example, in order to test the chip design, a test pattern is provided to perform a scan of the input and a check of the output result. Although this technology can minimize potential problems with the hardware design before the product is shipped, defect problems during operation that may be caused by the aging of the integrated circuit (IC), the external environment, and the fragility of the complex design may not be solved.

[0386] As mentioned above, the above-mentioned pre-deployment test cannot effectively solve hardware defects, which makes the inventors begin to be interested in runtime testing methods.

[0387] Pre-deployment testing and post-deployment testing may seem similar from a testing mechanism perspective, but there are clear differences in when testing can be performed. Specifically, pre-deployment testing can be performed only at certain times and is usually only allowed shortly after manufacturing. In contrast, testing during operation can be performed at any time under normal operating conditions.

[0388] There are two testing techniques for testing during runtime, including functional testing and scan testing.

[0389] According to the functional test, a test input is generated, and an output result obtained by inputting the generated test input into the original design is compared with an expected pattern. Alternatively, based on the original design, according to the functional test, input and output signals are monitored to detect abnormalities.

[0390] According to the scan test, an architecture for the scan test is inserted into the original design, and various test patterns as many as possible need to be created. As described above, after the scan architecture and the test pattern are prepared, the test during runtime can be performed in various ways.

[0391] To perform scan testing, ICT can connect multiple flip-flops in each CUT, import test inputs into at least one flip-flop, and obtain test results from the operation of the combinational logic of the flip-flops to analyze whether the CUT is defective or normal during runtime.

[0392] Figure 7 An example of a scan trigger is illustrated.

[0393] In order to design hardware more easily and minimize manufacturing defects, it is very important to apply design for testability (DFT).

[0394] To this end, the architecture for scan testing is reflected into the design, and a test coverage having a specific ratio of all detectable defects is defined to perform the test.

[0395] When using D-type flip-flops, the architecture for scan testing can be easily reflected in the design. During the test, all flip-flops in the CUT can be operated as scan flip-flops, which include D flip-flops and multiplexers.

[0396] With Figure 7 Compared to the ordinary D-type flip-flop shown in FIG. 1 , the flip-flop can use two additional pins, namely the scan enable (SE) pin and the scan input (SI) pin. The SI pin is used for test input, and the SE pin can be switched between the input for normal operation (D pin) and the test input (SI) for test operation.

[0397] Figure 8Illustrate an example of adding an architecture for scan testing in a hardware design.

[0398] like Figure 8 As shown, all SE pins in the scan flip-flops are connected to the scan_enable (SE) port, the SI pin of each flip-flop is connected to the Q pin of the previous flip-flop or the scan input port, and the Q pin of each flip-flop is connected to the SI pin of the subsequent flip-flop.

[0399] These connections create multiple scan chains. That is, the flip-flops are connected to each other to create scan chains.

[0400] When the SE (scan_enable) port is enabled, all scan flip-flops transfer data from the SI pin to the Q pin through the flip-flop, so the data can be transferred from the scan_in port to the corresponding scan_out port. All flip-flops on each scan chain transfer the test input from the scan_in port to the scan_out port.

[0401] The fewer flip-flops on a scan chain, the faster the data can be shifted. However, the number of flip-flops per scan chain and the number of scan chains are interdependent. The more scan chains you create, the fewer (smaller) flip-flops you have on each scan chain.

[0402] II. Testing through ICT

[0403] The above-mentioned test is performed as a background task, so that the test can be performed without reducing the system performance. Based on the monitoring of the operation of the component to be tested, the ICT can determine whether the component is in an idle state. When the component is in an idle state, the test is performed so that the system performance will not be degraded. The ICT continuously monitors the operating state of the CUT on the system bus, and the CUT may respond to unexpected access. When there is an access to the CUT, the operation of the CUT is switched from test operation to normal operation to restore the CUT, and the CUT is returned to normal operation. There may be a slight time delay in the switch. According to the present disclosure, the system bus can be effectively used during the time delay to minimize the degradation of system performance due to recovery.

[0404] II-1. Increased complexity of SoC or SiP architecture

[0405] The design of integrated circuits (ICs) is becoming increasingly complex and the level of integration is increasing significantly. SoC or SiP is a highly integrated semiconductor device, so that defects in certain functional components may lead to a degradation of the performance of the entire system. Therefore, it is becoming increasingly important to perform tests to find defects in functional components in SoCs.

[0406] From an operational perspective, Fig. 9A Pictured Figures 6A to 6C SoC or Figures 6D to 6F system.

[0407] Functional components (or IP) can be divided into three types: 1) internal processor, 2) interface or communication controller, and 3) memory. FIG9 shows a functional component (or IP) 100 / 200 / 300, an internal memory 400, a memory controller 450, a system bus 500, an ICT 600, a plurality of wrappers 700a, 700b, 700c, 700d, 700f and 700g (collectively referred to as 700), and an I / O interface 800.

[0408] Functional component (or IP) 100 / 200 / 300 can perform functions related to encoding, decoding, encryption, decryption and calculation. Functional component (or IP) 100 / 200 / 300 obtains raw data from internal memory 400 and processes the data with a specific algorithm. When the processing is completed, the output data can be sent together with a signal notifying completion.

[0409] The internal memory 400 may be a read-only memory (ROM) or a random access memory (RAM). The ROM corresponds to a nonvolatile memory, and the RAM corresponds to a volatile memory.

[0410] Volatile memory is a type of memory that stores data only when power is supplied, and the stored data is lost when power is turned off. Volatile memory may include static random access memory (SRAM) and dynamic random access memory (DRAM).

[0411] The internal memory 400 may include a solid state drive (SSD), a flash memory, a magnetic random access memory (MRAM), a phase change RAM (PRAM), a ferroelectric RAM (FeRAM), a hard disk, or a flash memory. The internal memory 400 may also include a synchronous random access memory (SRAM) and a dynamic random access memory (DRAM).

[0412] The I / O interface may support various protocols and functions to enable the SoC or SiP to communicate with various external hardware.

[0413] However, when the SoC or SiP is built into an autonomous system to be used, the SoC or SiP may be damaged due to aging of electronic components (e.g., transistors), physical impact, or usage environment. Specifically, when a functional component (or IP) in the SoC processes important data, the damaged functional component may generate erroneous output data, which may significantly reduce the accuracy of the autonomous system.

[0414] To prevent this problem, Fig. 9AAs shown, ICT 600 monitors system bus 500 and monitors the status of functional components 100 / 200 / 300, internal memory 400, memory controller 450 and I / O interface 800 via wrapper 700 or system bus 500. When a functional component in an idle state is found, ICT 600 selects the functional component as CUT to perform testing.

[0415] exist Fig. 9A In the figure, the connection to the system bus during normal system operation is represented by dotted lines, and the signals of the ICT and wrapper are represented by solid lines.

[0416] Fig. 9B The diagram shows the configuration used to test the NPU.

[0417] refer to Fig. 9B , NPU 100 may also include another component for testing the NPU. Fig. 9B At least one of the random number generator, the predetermined test data memory, and the temporary register may optionally be further included in Figure 1 3. The MUX may be provided between the NPU internal memory 120 and the processing element array 110 to perform internal testing of the NPU 100. The MUX may be configured to switch components configured to test the processing element array 110 and the NPU internal memory 120.

[0418] A method of testing the processing element array 110 using random numbers will be described. Fig. 9B The random number generator in the illustrated NPU 100 may generate random numbers based on a predetermined seed. The MUX selects at least one of the processing element arrays 110 to test whether the NPU 100 is defective.

[0419] The ICT 600 monitors the state of the NPU 100 via the wrapper 700 a , and when determining that the NPU 100 is in an idle state, the ICT 600 may command the NPU 100 to start a test.

[0420] Taking a specific example, the ICT 600 selects at least one of a plurality of PEs included in the NPU 100 to command the start of a test.

[0421] For example, when the ICT 600 determines that a predetermined percentage of PEs (e.g., 20% of all PEs) among the plurality of PEs included in the NPU 100 are in an idle state, the ICT 600 may instruct the NPU 100 to start a test. In other words, when the proportion of idle PEs among all PEs is greater than or equal to a threshold, the ICT may instruct the start of a test.

[0422] Taking a specific example, when the ICT 600 selects a predetermined percentage of PEs (eg, 50% of all PEs) among a plurality of PEs included in the NPU 100 to instruct the NPU 100 to start a test.

[0423] When performing the test of the NPU 100, the inference speed of the NPU, i.e., the inferences per second (IPS), may be reduced. Specifically, the inferences per second may be reduced according to the number of PEs to be tested. For example, when 50% of all PEs are tested, the inferences per second may be reduced by about 50%, and when 30% of all PEs are tested, the inferences per second may be reduced by about 30%.

[0424] Therefore, according to an example, the NPU 100 may also include additional PEs to improve the speed reduction due to testing.

[0425] As another example, when the NPU 100 operates at a value lower than a predetermined IPS value, the ICT 600 may instruct the NPU 100 to perform a test. Specifically, when it is assumed that the NPU 100 operates at a maximum of 100 IPS, and the IPS threshold is 30 IPS, if the NPU 100 operates at 30 IPS or higher, the ICT 600 may instruct the NPU 100 to perform a test during the remaining time. For example, when the NPU 100 operates at 40 IPS, the remaining 60 IPS may be used for testing. Therefore, a significant speed reduction of the NPU is not caused.

[0426] As another example, when data transmitted to the NPU internal memory 120 in the memory 400 is delayed so that the NPU 100 is in an idle state or enters a data shortage period, the ICT 600 may instruct the NPU 100 to perform a test.

[0427] When testing the NPU 100, the register file RF corresponding to each PE in the NPU 100 is initialized to predetermined test input data, and the corresponding PE can perform reasoning according to the test input data in the register file RF. The predetermined test input data can be a functional test or a partial functional test of the NPU.

[0428] When the NPU 100 is tested, as described above, the random number generator in the NPU 100 generates random numbers. By doing so, the register file RF is initialized with the generated random numbers, and the corresponding PE performs reasoning according to the random numbers in the register file RF.

[0429] Instead, the ICT 600 instructs the CPU 200 to import the test input data into the register file RF in the NPU 100 via the wrapper 700 b .

[0430] When the NPU 100 is tested, multiple register files RF in the NPU 100 are initialized to a single test input data, and the corresponding PEs can perform reasoning according to the test input data in the register file RF. Specifically, multiple PEs in the NPU 100 can be tested based on the same single test input data, and the reasoning results are output.

[0431] Alternatively, when the NPU 100 is tested, some register files RF in the NPU 100 are initialized based on specific test input data, and corresponding PEs may perform inference according to the test input data in the register files RF.

[0432] The register file RF may reset the flip-flops in each PE and transfer the test input data to the PE as described above.

[0433] For example, the size of each RF may be 1 Kb.

[0434] II-2. Necessity of packaging

[0435] Fig.10 The operation of the wrapper is illustrated.

[0436] As described above, ICT can test many functional components (that is, IP, I / O interface, memory, etc.) in a SoC or SiP during its runtime. To this end, during the test of the functional component selected as the CUT, it is necessary to resolve the conflict problem caused by accessing the functional component from the system bus.

[0437] In order to resolve the conflict problem, after monitoring whether the functional component is in an idle state, when the functional component is monitored to be in an idle state, the functional component is switched from the normal operation mode to the test operation mode, and then the test needs to be performed. When a conflict is detected during the test, the functional component needs to be switched to the normal operation mode. After switching the operation to the normal operation mode, the functional component needs to correctly process the input data.

[0438] To this end, the wrapper 700 shown needs to be provided between the functional components and the system bus 500. The wrapper 700 may include a multiplexer gate that selectively controls the input and output of each operation mode.

[0439] As shown in the figure, when the TEST_ENABLE port is turned on, the test vector can be input to the CUT, and the TEST_OUTPUT port can send the output. The general data output from the wrapper 700 can be transmitted to other functional components through the system bus. In contrast, the test results can be directly transmitted to the ICT 600. The ICT 600 can receive the test vector for testing from the external memory or the internal memory, and store the test results in the internal memory or the external memory, or transmit the test results to the outside.

[0440] In order to test SoC or SiP at runtime, ICT 600 can perform multiple processes. First, ICT 600 can select the functional component to be tested as CUT based on predetermined rules. Since CUT needs to respond to access from the system bus when SoC or SiP is in operation. Therefore, it is effective to select as many functional components in idle state as possible as CUT. To this end, ICT 600 can monitor whether the functional component enters the idle state. When the functional component enters the idle state, wrapper 700 can open the TEST_ENABLE port. ICT600 can import test vectors into CUT through the TEST_ENABLE port.

[0441] ICT 600 can collect and analyze test results from CUT via the TEST_OUTPUT port of wrapper 700. When the test results indicate that a problem is detected, ICT 600 can perform post-action. During testing, when general access to CUT from system bus 500 is detected, ICT 600 can temporarily delay access from system bus 500 and then immediately stop (interrupt) the test operation. Thereafter, ICT 600 can restore the previous value of the register setting for CUT and close the TEST_ENABLE port of wrapper 700. When normal operation of CUT is ready, ICT 600 can control wrapper 700 to return the connection for input and output with CUT to system bus 500.

[0442] Fig.11 The diagram shows the internal configuration of the ICT.

[0443] refer to Fig.11 , the ICT 600 may include a configuration data (CONF_DATA) restorer 610 , a state detector 620 , a scheduler 630 , a tester 640 , a test vector generator 650 , a host interface 660 , and a post-action (POST_ACT) unit 670 .

[0444] The state detector 620 can detect whether the functional components in the SoC chip are in an idle state or a busy state (or a processing state). When any functional component enters an idle state, the state detector 620 sends the ID (C_ID) of the functional component to the scheduler 630 to perform a test.

[0445] The scheduler 630 may manage the overall operation of the ICT 600. The scheduler 630 may receive the status of the functional components from the status detector 620 and trigger the test. The scheduler 630 may send the ID of the component to the tester.

[0446] The tester 640 controls the wrapper 700, transmits the test vector, obtains the test result, and then compares whether the test result matches the expected test result. Thereafter, the tester 640 may transmit the test result to the post-action unit 670. The tester 640 may restore the register settings of the functional component selected as the CUT to its original value.

[0447] The test vector generator 650 can generate test vectors (or predefined test input data) and corresponding expected test results. The test vector generator 650 can include a buffer, a memory interface, a memory for storing test vectors and expected test results, and a random number generator. When the test starts, a test pattern for generating a test vector can be loaded in the buffer. The random number generator can be used to generate a test vector. The random number generator can make the memory not store all test vectors, but generate various test vectors.

[0448] The post-action unit 670 may perform a post-action when receiving an ID (eg, C_ID) of a functional component found to have a problem from the tester 640. The post-action may isolate the defective functional component or notify a user or a remote host device of the defect.

[0449] The host interface 660 can report to the user or remote host device the functional components found to have problems during the test process. If there are changes related to the test operation, the host interface 660 can notify the remote host device.

[0450] When the test is completed or an access to a functional component selected as the CUT is detected from the system bus 500 during the test, the configuration data restorer 610 can restore the register settings of the CUT so that the tester 640 can switch the CUT to the normal operation mode. Most functional components will have specific register setting values ​​for normal operation. Accordingly, the configuration data restorer 610 can store the register setting values ​​of the functional component before performing the test, and restore the register setting values ​​to the functional component when it is necessary to switch the CUT to the normal operation mode.

[0451] II-3. Detecting the idle state of functional components

[0452] Fig.12 The diagram illustrates the operation of monitoring whether a functional component is in an idle state through ICT.

[0453] In order to detect whether the functional component is in an idle state during the normal operation mode, the ICT 600 can use one or both of two technologies. First, the ICT 600 can monitor whether the component is in an idle state or in use based on some hardware signals that directly or indirectly indicate whether it is operated. For example, the ICT 600 can monitor the power gating control signal to disconnect the connection of the functional component, thereby reducing the power consumption of the functional component. In addition, the ICT 600 can determine whether the functional component is in an idle state based on the output signal that directly or indirectly indicates whether the component is operated or the value of the register that stores information related to the operation in the functional component. Secondly, the ICT 600 monitors the signal from the system bus via the wrapper 700 or monitors the input / output port of the functional component during a specific time period to determine whether the functional component is in an idle state.

[0454] II-4. Handling of access conflicts

[0455] Fig.13 The diagram illustrates the operations between a master, a slave, and an arbiter operating on a system bus.

[0456] A master on a system bus may be an entity that uses a slave, a slave may be an entity that is used by a master, and an arbitrator may be an entity that arbitrates and decides between a master and a slave.

[0457] Fig.13 The slave shown in may be a functional component selected as a CUT and the arbitrator may be an ICT.

[0458] When access to normal operation is detected from the system bus 500 while testing a functional component selected as the CUT, the ICT 600 may require a predetermined amount of time or more to restore the CUT to its previous state. The ICT 600 can temporarily deactivate (or deassert) the HREADY signal to temporarily stop system access from the host, stop (interrupt) the test activity, restore the register settings of the CUT, and change the direction of data input to or output from the wrapper. When the CUT as a slave is ready to perform a task with the host, the HREADY signal can be turned on. However, according to the present disclosure, the ICT may cause some time delays in the bus separation operation. The specific process will be described below.

[0459] First, the host activates (or asserts) the HBSREQ signal for bus access. Second, during the arbitration or determination process, the arbitrator activates (or asserts) the HGRANT signal to allow bus access. By doing this, the host can transfer data to the CUT as a slave through the system bus. If the ICT is performing the processing operation of the test, the ICT sends the HSPLIT signal to the arbitrator together with the bit indicating the current host, and simultaneously activates (or asserts) the SPLIT signal in the HRESP signal. After activation (assertion), the host cancels access to the CUT, and the arbitrator performs the arbitration or determination process without host intervention. When the CUT is ready to respond to access from the host, the ICT deactivates the HSPLIT signal, and the host waits for authorization from the arbitrator to resume the task of accessing the CUT.

[0460] Fig.14 An example of adding a shift register in a SoC chip or SiP is shown.

[0461] The inventors of the present disclosure have recognized that access to an I / O interface may not cause a conflict on the system bus. For example, when the target CUT is a master, an external device connected through the I / O interface will not request access for itself, and thus no conflict will occur. Therefore, it may be effective to focus only on resolving the conflict problem that occurs when the CUT is a slave.

[0462] Instead, in order to delay the data transferred from the external device to the CUT during the recovery time, a shift register can be added between the port of the SoC and the external interface port of the CUT.

[0463] A shift register can be added to store the access signal input from outside the SoC while the CUT is being restored. When the CUT is ready, the access signal is regenerated by the shift register for output.

[0464] The depth of the shift register can be determined by the number of clock cycles required to restore the CUT to normal operation. Specifically, when one or more functional components need to receive signals from outside the SoC, the depth of the shift register can be variable. In this case, the depth of the shift register can be determined by the ICT.

[0465] II-5. Operation sequence of ICT

[0466] Fig.15 The diagram illustrates the sequence of operations of ICT.

[0467] refer to Fig.15 , when a timer associated with the start of a test of the ICT in runtime expires (S601), the ICT monitors whether any functional component is in an idle state and detects the functional component in the idle state (S603).

[0468] By doing so, the ICT performs a test preparation process (S605). The test preparation process may include selecting a functional component as a CUT, isolating the functional component selected as the CUT from the system bus, and generating a test vector as test input data. Isolation from the system bus may mean that the ICT changes the direction of inputs and outputs on a wrapper that communicates with the functional component selected as the CUT.

[0469] The ICT imports the test vector as the test input data into the CUT (S607).

[0470] When the test is completed normally, the ICT checks the test result (S609). To check, the ICT can compare whether the test result matches the expected test result.

[0471] When the test result indicates that the functional component selected as the CUT has no problem (that is, has no defect or damage), the ICT may restore the functional component to a normal operating state (S611).

[0472] Meanwhile, when an access to the functional component selected as CUT is detected from the system bus during test preparation or testing, the ICT may restore the functional component selected as CUT to a normal operating state (S613). Restoration may mean that the register setting value of the functional component selected as CUT is restored and the direction of input and output is returned to the original state on the wrapper communicating with the functional component selected as CUT.

[0473] In this case, the ICT drives a back-off timer (S615) and when the back-off timer expires, it may return to step S603.

[0474] Meanwhile, when the test result indicates that the functional component selected as the CUT has a problem (that is, is defective or damaged), the ICT may perform a post-operation (S617).

[0475] II-6. Internal memory test

[0476] Fig.16 Diagram shows the internal memory testing process.

[0477] The test of internal memory may be different from the test of functional components. In the following, two testing techniques of internal memory will be proposed.

[0478] The first technique is a technique for detecting errors using an error detection code in the process of reading data from an internal memory. If the error detection code acquired during the reading process is different from a predetermined error detection code, the ICT can determine the code as an error.

[0479] The second technique is a technique of performing read and write tests in a hard manner during normal operation.

[0480] Fig.16 The second technique is shown in the figure. The test logic of the encapsulated internal memory can perform read and write tests while the system is running and bypass access from the system bus. In order to handle the test completely, the tester in the ICT can be responsible for address management. The temporary register file shown in the figure can temporarily store the original data that is easily deleted due to the test. After the test is completed, the original data in the temporary register file is recorded again in the internal memory.

[0481] If an unpredictable access occurs during the test, the data on the system bus can be recorded in the temporary register file, and conversely, the data in the temporary register file can be moved to the system bus.

[0482] The testing technique described above can be applied not only to internal memory but also to external memory in the same manner.

[0483] II-7. Post-test operations

[0484] When a hardware defect exists in an SoC or SiP, the post-test action may be very important. For example, notifying the user of the defect to recommend discontinuation of use. Fig.11 The post-action unit 670 can provide information about the detected defective functional component and information about the test input data (that is, the test vector) that caused the defect. The above information can enable the user to know the location of the defective functional component. It is necessary to stop and isolate the use of the detected defective functional component. In order to prevent the defective functional component from reducing the performance of the entire system, the output signal of the functional component can be replaced with a predetermined signal. Alternatively, the functional component can be reset or gated. Alternatively, power gating can be performed on the functional component.

[0485] At the same time, when functional components are isolated, SoC or SiP faces another problem. Therefore, even if some functional components are defective, it is necessary to propose a method that allows SoC or SiP to still operate. For example, when SoC or SiP is installed in a product that requires high reliability, SoC or SiP also needs to include spare parts for some functional components. If some functional components are defective, the spare parts may operate instead of the functional components. However, when some functional components are duplicated, the area of ​​the semiconductor device may increase. To solve this problem, it may be effective to add programmable logic to SoC or SiP.

[0486] III. Functional testing or functional combination testing during SoC or SiP runtime

[0487] Fig.17 Diagram showing the process of testing a function using a random number generator.

[0488] Functional testing is a test that imports test input data (e.g., test vectors) into the CUT and compares whether the output from the CUT matches the expected output. In order to correctly evaluate based on the comparison, each input data needs to accurately induce the expected output. The test coverage of the test input data needs to be large to detect all defects.

[0489] In a specific design, there can be two test input data for functional testing. First, the random number generator connected to the XOR operation can be used Fig.17 The test operation shown. Generally, a random number generator can generate a pseudo-random number stream based on an input seed. The random number stream is imported into the CUT through a wrapper, and the output is accumulated and stored in a test result register through an XOR operation. When the test is completed, the value stored in the test result register can be compared with the expected result corresponding to the test input data. If there is a difference in the comparison result, an error notification may be issued.

[0490] Secondly, all test patterns of the test input data and the corresponding prediction results can be fixed respectively and stored in the internal memory of the SoC or SiP or in the external memory. When the test input data (that is, the test vector) from the memory is input to the CUT, the output of the CUT can be compared with the expected result corresponding to the test input data.

[0491] In order to perform functional testing during the runtime of SoC or SiP, ICT plays an important role in transmitting data, communicating with the system bus, monitoring the status of CUT, etc. Specifically, when the CUT is in an idle state, ICT needs to determine when to test. During the test process, the random number generator generates a random number stream as the test input data and transmits the test input data to the CUT. If there is a difference between the test result and the expected test result, ICT transmits the information to the post-operation unit.

[0492] During functional testing, functional components in the SoC may be used so that, in general, the frequency of the test operation needs to be lower than or equal to the frequency of normal operation to avoid timing differences (i.e., timing violations). In order to perform testing in real time during normal operation, it is effective to perform testing when the functional components are in an idle state. Therefore, there is no choice but to perform testing at a high frequency.

[0493] IV. Use a combination of DFT (Discrete Fourier Transform) and ICT to test at SoC or SiP runtime

[0494] IV-1. Multiple clocks

[0495] Fig.18A An example of multiple clocks is illustrated. Fig.18B is an example diagram illustrating the operation of the tester at multiple clocks, and Fig. 18C Diagram showing the path of test input data.

[0496] During the testing process, there are two techniques for importing test input data (ie, test vector).

[0497] The first technique is to use time periods to Fig.18A The "Move Data" is shown. The SE (Scan Enable) port is enabled, and the Q output of the flip-flop is connected to the D input of another flip-flop. This connection can form a scan chain that connects the scan input to the scan output through a chain of flip-flops.

[0498] Therefore, all of the design's combinatorial logic may be disabled, and there may be no reference logic cells for the data paths (that is, the path from one flip-flop to another).

[0499] When T cycle Defined as the clock period, T launch Defined as the time delay from the clock source of the first flip-flop to the CK pin, T capture Defined as the time delay from the clock source to the CP pin of the second flip-flop, T clk2q Defined as the time delay from CK to Q pin of the first flip-flop, T dpmax Defined as the time delay from the Q of the first flip-flop to the D of the second flip-flop, T cycle >T launch +T clk2q +T dpmax +T setup +T margin –T capture .

[0500] When scan testing is enabled, from the perspective of scan testing, T dpmax can be reduced to zero. Ideally, T dpmax May be zero. However, to resolve timing violations, when multiple inverters or buffers are added, the time delay may be greater than zero.

[0501] As an alternative, T dpmax >>T clk2q +T setup +T launch –T capture During the "moving data" time period, it is processed at a higher frequency.

[0502] In such Fig.18ADuring the "Capture Data" time period shown, the scan enable pin is disabled and therefore the functional components are re-enabled and the combinational logic can be enabled on the data path. To resolve timing violations when capturing data, a time delay can be added between the clock at one end of the "Shift Data" time period and the clock at one end of the "Capture Data" time period.

[0503] The delay between clock cycles can be greater than or equal to the clock cycle of normal operation. To detect when the "shift data" period is complete based on the maximum number of flip-flops on the scan chain corresponding to the shift value, a counter is added, and to manage the time delay within the "capture data" period, another counter can be added.

[0504] exist Fig.18B In the test block, the test block receives two input clocks. One is f_clk for normal operation and the other is sclk for "shifting data". Insert the "clock configuration" into the tester block so that the s_clk signal can be set to be used during the "shifting data" period and the "capturing data" period.

[0505] To control the switch between f_clk for normal operation and s_clk for test operation, the TE signal corresponding to CUT can be used. When the ID of the component (i.e. C-ID) is received from the scheduler, the test block in ICT is ready for testing. The TE of CUT available through the decoder can enable the test process.

[0506] Fig.19A An example of a functional component is shown, Fig.19B An example is illustrated in which test input data (eg, test vectors) are imported into a tester in an ICT.

[0507] In order to apply Discrete Fourier Transform (DFT) in test during the runtime of SoC or SiP, scan chains are added in CUT and all flip-flops can be surrounded by scan flip-flops. Scan input, scan output, and TEST_ENABLE and SCAN_ENABLE signaling are connected to the tester in ICT, and the raw input and raw output of CUT can communicate with the system bus through the tester and wrapper.

[0508] like Fig.19B As shown, from the perspective of the memory storing the test pattern, the block can be divided into four parts. The first part is the part storing the input shift vector, the second part is the part storing the output shift vector, the third part is the part storing the input capture vector, and the fourth part is the part storing the output capture vector. To start the test, the input shift data is loaded from the memory to be input to the CUT through the tester.

[0509] In each scan chain, after all flip-flops are filled with shift vectors, when the first input capture vector including the values ​​of the scanned inputs and the values ​​of the initial inputs is loaded, the first output capture vector including the values ​​of all scanned outputs and the values ​​of the initial outputs is loaded and then compared with the actual output capture data. Each loaded shift vector is accompanied by output shift data, and the actual output data and the output shift vector or the output capture vector can be compared.

[0510] Fig. 20 Diagram shows the testing process using DFT. Fig.21 Examples of shifted data and captured data during a testing process are illustrated.

[0511] During the data shifting step, when the scan_enable port is enabled, the SCAN_IN port can be connected to the SCAN_OUT port through a flip-flop without combinatorial logic. The input shift vector can be loaded in all scan chains until all flip-flops have values ​​shifted from the input shift vector. One shift value can pass through one flip-flop per clock cycle. That is, the D pin of the previous flip-flop can be connected to the D pin of the subsequent flip-flop.

[0512] When the scan_enable port is disabled during the capture step, the D pins of all flip-flops are not connected to the Q pins of the previous flip-flops, but can be directly connected to the combinational logic.

[0513] The capture vector output can be loaded into the Q output of all flip-flops through combinatorial logic at the positive (+) edge of the clock cycle. In the first data capture step, the data transfer process is prepared to compare the output data with the expected output data, and then the comparison is performed at each positive clock edge. All test vector inputs are loaded, the process returns to the first data shift step, and each process starts over.

[0514] Fig.21 The displacement and capture process is illustrated. Fig.21 The rectangular boxes in represent the flip-flops in each scan chain, and all flip-flops are filled at the end of the data shifting step.

[0515] Fig. 22 An example of switching the test mode to the normal operation mode is illustrated.

[0516] As reference Fig. 22 As is known, the data shifting process and the capture step can be repeated during the output test mode. If there is an access to the CUT, the CUT will be restored to the normal operating mode and the test will be rolled back. Thereafter, a skip mode is performed during a predetermined period of time and then the output test mode can be performed again.

[0517] Fig.23 illustrates an example of a flip-flop operating on a scan chain, and Fig.24 A portion of the CUT operating in a normal operating mode is illustrated.

[0518] When an unexpected access to the CUT from the system bus occurs, TEST_ENABLE is disabled and data shifting or capturing can be stopped quickly. The CUT is restored to normal operating mode and the test can be rolled back.

[0519] When the CUT enters the idle state again, the previous data shifting step can be restarted for testing. However, in the first switching step after switching from the normal operation mode to the test operation mode, the comparison of the output results is disabled and the comparison of the output results can be performed from the subsequent capture step.

[0520] That is to say, Fig.23 As shown, the shifted input value is not loaded into all flip-flops in the scan chain, so comparison is not required.

[0521] V. Simulation

[0522] To verify the above, simulations from simple sub-cases to complex sub-cases were performed using electronic design automation tools from Synopsys, Inc. When verification is successfully performed, the design implemented by the software code can be converted into logic gates. As a next step, all D flip-flops (DFFs) can be replaced by scan flip-flops and scan chains can be generated. In order to increase the scope covered by the test, the netlist can be modified and tested repeatedly. ATPG tools can be used to generate test vectors and expected results. When the scan is ready and the test pattern of the netlist is inserted, ICT can be applied to the design. The details of each test case will be described with reference to the accompanying drawings.

[0523] Fig.25 The process used for the simulation is illustrated.

[0524] The simulation process shown uses the Design Compiler tool from Synopsys, Inc. The Design Compiler (DC) can also be used to convert software code to logic gate level based on timing constraints (e.g., period, transition, capacitance, timing information contained in the library package). When all constraints are met, value optimization can be repeated. When the design does not meet the requirements, the constraints can be adjusted.

[0525] The output of the DC tool, including the above timing constraints, can be used as input for design for test (DFT).

[0526] In the scan test import step, the number of scan ports and scan chains can be set. Usually, in order to minimize the extra ports during design, the scan ports are generated using the original input and output ports. In addition, the number of input scan ports is equal to the number of scan chains. The more the number of scan chains, the fewer the number of shift clock cycles used to shift the data. Therefore, maximizing the number of scan chains is the best choice for testing. When the scan setting is completed, the DFT compiler can replace all the flip-flops with flip-flops for scan, and connect the scan input (scan_in) pin and the scan output (scan_out) pin to each other to generate the scan chain. The additional connections and scan flip-flops make the design more complex and cause time delays for most of the data paths, so the DFT compiler can continuously optimize power and timing after connecting the scan chain. After the scan test import is completed, the DFT DRC that follows the DFT rules checks whether all test connections are connected. The test input data (i.e., test vectors) are ready.

[0527] To check the test coverage and generate test patterns, the output of the DFT compiler is input to Tetramax from Synopsys, Inc. When the test coverage does not meet the expected requirements, the scan test introduction step can be performed again to modify the design. This task can be repeated until the desired test coverage is obtained.

[0528] V-1. Design experiment

[0529] As a design experiment, a JPEG image encoder can be used. The test can use approximately 265,118 combinational cells, 72,345 continuous cells, and 31,439 inverter / buffer cells. As a result of execution based on cell library information and many thresholds, it is confirmed that the frequency that meets the timing constraints is 100MHz and the frequency for the mobile test mode is 1GHz. About 512 scan chains are used, and the maximum number of triggers used on each scan chain is 75. Therefore, it is confirmed that the time period for the mobile test mode is approximately 75 cycles, which corresponds to 75ns. It takes one cycle to capture data, which corresponds to approximately 10ns. About 256 test patterns are input, each test pattern includes approximately 75 test vectors for shifting, and can be captured using approximately one test vector. In order to complete one test cycle, 13,260 (156×75+156×10)ns are required.

[0530] Fig.26 Diagram showing the test architecture for the JPEG image encoder.

[0531] As mentioned above, in order to check whether the ratio of controllable nodes and observable nodes is sufficient for testing, Tetramax is used to confirm the test range. Generally, 99.97% of the test range and 1,483.5K bytes are used for testing.

[0532] From the perspective of power consumption, each test is measured by different types of inputs. In Table 1 below, the internal power, switching power, and leakage power of each input are represented. Specifically, the input that cannot be controlled for test mode and normal mode is first implemented. This is called static power measurement. After input to the power compiler tool, in order to estimate the power consumption in test mode, the TE (test_enable) port is turned on, and in order to estimate the power consumption in normal working mode, the TE (test_enable) port is turned off. Secondly, in order to estimate the power consumption, the input is controlled by a specific time interval. This is called dynamic power estimation. The control of the input can be divided into three modes. First, turn on the test mode and provide the test mode. Second, turn off the test mode and provide the input for normal operation. Third, after completing the test of each test mode, switch the mode to the normal operation mode. In order to obtain its power consumption value, the test mode and the normal operation mode are switched.

[0533] The following table shows the power consumption values ​​of the JPEG image encoder.

[0534] Table 1

[0535]

[0536] Similar to the JPEG image encoder, the Advanced Encryption Standard (AES) functional components were tested. In addition, the functional components used for image classification in self-driving cars were also tested. The results are shown below.

[0537] Table 2 below represents the AES design.

[0538] Table 2

[0539] Number of combined units 160,261 Number of consecutive units 11,701 Number of buffers / inverters 22,377 All regions 400,464.797234 Normal operating mode frequency 100Mhz Test mode frequency 1Ghz Moving time 46ns Capture time 10ns Number of test patterns 315 Test range 100％ Memory size 948.5KB

[0540] Table 3 below represents the testing of the functional components of AES.

[0541] Table 3

[0542]

[0543] Table 4 below shows the details of CONVO2.

[0544] Table 4

[0545] Number of combined units 2,245,932 Number of consecutive units 424,695 Number of buffers / inverters 154,510 Normal operating mode frequency 50Mhz Frequency of test mode 1Ghz Moving time 829ns Capture time 20ns Number of test patterns 183 Test range 100％ Memory size 18.634MB

[0546] Table 5 below shows the power consumption of CONVO2.

[0547] Table 5

[0548]

[0549] Functional testing and testing by scan introduction have their own advantages and disadvantages. Testing by scan introduction has the disadvantage of using more memory and time delay compared to functional testing, and has the advantage of a wide test range.

[0550] Specifically, when the SoC or SiP is installed in a product that requires high reliability, such as self-driving cars, drones, urban air traffic (UAM), and unmanned aerial vehicles (UAV), scan-introduction type testing with a wide test range will be advantageous. In addition, scan-introduction type testing can increase the frequency of test operations and reduce test time. When a long time is required for testing, the possibility of a car accident may increase, making it undesirable. Scan-introduction type testing increases the frequency of test operations, so that more test modes can be introduced during idle time, and hardware defects in the SoC can be detected faster. The advantage of ordinary functional testing is low power consumption, but in environments requiring high reliability, such as self-driving cars, drones, urban air traffic (UAM), and unmanned aerial vehicles (UAV), power consumption is not important.

[0551] So far, SoC has been mainly explained, but the disclosure of this specification is not limited to SoC, and the disclosure is also applicable to a board-level system based on a system-in-package (SIP) or a printed circuit board (PCB). For example, each functional component is implemented by an independent semiconductor chip and connected through a system bus, which is implemented by a conductive pattern formed on a PCB.

[0552] The embodiments of the present disclosure disclosed in this specification and the accompanying drawings only provide a specific example to facilitate description and better understanding of the technical description of the present disclosure, and are not intended to limit the scope of the present disclosure. It is obvious to those skilled in the art that, in addition to the examples described so far, other modifications are also possible.

Claims

1. A system capable of testing components in the system during runtime, the system comprising: substrate; a plurality of functional components mounted on the substrate and including circuits; a system bus formed on the substrate using a conductive pattern to allow the plurality of functional components to communicate with each other; one or more wrappers, each of the one or more wrappers connected to one of the plurality of functional components; and An In-System Component Tester (ICT) configured to: selecting at least one functional component in an idle state among the plurality of functional components as a component under test (CUT); as well as The at least one functional component selected as the CUT is tested by the one or more wrappers.

2. The system according to claim 1, wherein: The ICT is further configured to: Status of the plurality of functional components is monitored via the one or more wrappers.

3. The system according to claim 1, wherein: The ICT is further configured to: stopping the test with respect to the at least one functional component selected as the CUT based on detecting a conflict with access from the system bus to the at least one functional component selected as the CUT; as well as Based on the stopping, connection of the at least one functional component is allowed to return to the system bus.

4. The system according to claim 3, wherein: The ICT is further configured to return to the selecting step if the at least one functional component is in the idle state again after the connection of the at least one functional component is returned to the system bus.

5. The system of claim 4, wherein returning to the selecting step occurs after a backoff timer regarding the conflict expires.

6. The system of claim 1, wherein the plurality of functional components comprises: One or more Universal Processing Units (UPUs).

7. The system of claim 6, wherein the UPU comprises at least one of the following: one or more central processing units (CPUs); one or more graphics processing units (GPUs); and One or more neural processing units (NPUs) configured to perform operations of an artificial neural network (ANN) model.

8. The system according to claim 6, wherein the plurality of functional components further comprises at least one of the following: at least one memory; at least one memory controller; and At least one input and output (I / O) controller.

9. The system according to claim 1, wherein: The ICT is further configured to: For the testing step, the one or more wrappers are instructed to isolate a connection of the at least one functional component selected as the CUT from the system bus.

10. The system of claim 1, wherein the ICT comprises at least one of the following: a detector configured to monitor the status of a plurality of functional components; a scheduler configured to manage the operation of the ICT; a generator configured to generate test input data; and A tester is configured to import the test input data into the CUT and analyze a test result obtained from the CUT processing the test input data.

11. The system according to claim 10, wherein: The test input data is predefined test data or a random bit stream generated based on a seed.

12. The system of claim 1, wherein the ICT is further configured to: After the testing step is completed, analyzing the test results obtained from the at least one functional component selected as the CUT; and Based on the at least one functional component being analyzed as normal, connection of the at least one functional component to the system bus or another system connection is permitted.

13. The system of claim 1, wherein the ICT is further configured to disable the at least one functional component based on the at least one functional component being analyzed as defective.

14. The system of claim 11, further comprising: A field programmable gate array (FPGA) is configured to emulate the at least one functional component analyzed as defective.

15. The system of claim 14, wherein the address of the field programmable gate array is deprecated and replaced by the address of the at least one functional component analyzed as defective.

16. The system of claim 13, wherein the deactivating step includes revoking an address of the at least one functional component analyzed as defective.

17. The system of claim 13, wherein: The deactivating step includes powering off the at least one functional component analyzed as being defective.

18. The system of claim 13, wherein: The deactivating step includes isolating the at least one functional component analyzed as defective from the system bus by cutting off the system bus connection with the at least one functional component analyzed as defective.

19. The system according to claim 13, in, The plurality of functional components include a spare component for the at least one functional component analyzed as defective; and Wherein, the ICT is further configured to enable the backup component.

20. The system according to claim 1, The test steps include scan testing which is different from functional testing, and in, For the scan test, the ICT is further configured to: Connect multiple triggers in each CUT to each other. direct the test input into at least one trigger, and A test result is obtained from the operation of the combinatorial logic of the flip-flop to analyze whether the CUT is defective or normal during run-time.

21. The system of claim 1, wherein the plurality of functional components comprises a neural processing unit (NPU) comprising a plurality of processing element arrays and configured to select and test at least one processing element in the plurality of processing element arrays.

Images