Electronic devices and their control methods

Abstract

An electronic device is provided. The electronic device includes: a memory configured to store first information and second information; and a processor configured to obtain a first weight matrix by loading the first information, or to obtain a second weight matrix by loading the first information and the second information, based on resource information of the electronic device. The first information includes weights associated with the first weight matrix and a first index corresponding to the weights, and the second information includes additional weights for obtaining the second weight matrix and a second index corresponding to the additional weights.

Description

Technical Field

[0001] The apparatuses and methods consistent with this disclosure relate to electronic devices and control methods thereof, and more specifically, to electronic devices and control methods thereof that perform weight reduction operations in neural networks. Background Technology

[0002] When executing deep neural networks (DNNs) in resource-constrained electronic devices, neural network weight reduction techniques—which transform existing neural networks into smaller (simpler) neural network models—are crucial. This is because reducing the weights of a neural network reduces the size of the parameters, thereby reducing power consumption and memory bandwidth due to transmission, and significantly reduces computational load by using compressed parameters for computation.

[0003] Most DNN operations are configured as products of inputs (features) and weights (or parameters), and forcing some parameters to zero is called pruning. When applying pruning, only parameters with non-zero values ​​can be compressed to reduce data transfer, and parameters with zero values ​​can be excluded from computation, significantly reducing computational cost. However, as the parameter values ​​change during training, it may differ from the actual accuracy. Therefore, a retraining process can restore the approximate accuracy before pruning. Summary of the Invention

[0004] Technical issues

[0005] As mentioned above, power consumption and computation can be reduced by using neural network weight reduction operations. However, in related technologies, once a neural network weight reduction method is determined, there is a problem that the determined weight reduction method is only operated on and cannot be scalable. Specifically, there is a problem that neural network weight reduction is uniformly applied according to the resources of the electronic device, and because the weight reduction method is uniformly applied according to various situations of the electronic device, efficient operation is not possible.

[0006] Solution to the problem

[0007] The embodiments of this disclosure overcome the above-described disadvantages and other disadvantages not described above. Furthermore, this disclosure does not claim to overcome the above-described disadvantages, and the embodiments of this disclosure may not overcome any of the above-described problems.

[0008] This disclosure provides an electronic device and a control method thereof for performing neural network weight reduction operations by taking into account the resource information of the electronic device.

[0009] According to one embodiment of this disclosure, an electronic device includes: a memory configured to store first information and second information; and a processor configured to obtain a first weight matrix by loading the first information, or to obtain a second weight matrix by loading the first information and the second information, based on resource information of the electronic device, wherein the first information includes weights associated with the first weight matrix and a first index corresponding to the weights, and the second information includes additional weights for obtaining the second weight matrix and a second index corresponding to the additional weights.

[0010] The processor can be configured to: obtain output data by inputting input data into an artificial intelligence model using a first weight matrix or a second weight matrix, and obtain a second weight matrix based on the first weight matrix or a first weight matrix based on the second weight matrix, based on changes to the resource information of the electronic device.

[0011] The second weight matrix can be obtained based on the weights and first index included in the first information, the additional weights, and the second index.

[0012] The processor can be configured to obtain a second weight matrix by combining the additional weights with the weights included in the first information based on the first and second indices.

[0013] The processor can be configured to: determine the position of the weight corresponding to the first index in the second weight matrix based on the first index; determine the position of the additional weight corresponding to the second index in the second weight matrix based on the second index; change at least one of the first index or the second index based on the position of the weight corresponding to the first index and the position of the weight corresponding to the second index; and obtain the second weight matrix based on the position of the weight, the position of the additional weight, and the changed index.

[0014] Resource information may include at least one of the following: the available capacity of the electronic device's internal memory, the power status of the electronic device, the resources of an application running in the electronic device, or the resources of an artificial intelligence model running in the electronic device.

[0015] The first information may include a second threshold or a greater weight in the information obtained by pruning the weights less than a first threshold in the weight information, and an index corresponding to the weights less than the second threshold or a greater weight. The second information may include weights less than a second threshold in the information obtained by pruning, and an index corresponding to the weights less than the second threshold.

[0016] The processor can be configured to modify the first information by performing first training based on the first information, modify the second information by performing second training based on the first information and the second information, and obtain a first weight matrix by loading the modified first information or obtain a second weight matrix by loading the modified first information and the modified second information.

[0017] The processor can be configured to obtain the first weight matrix by erasing additional weights corresponding to the second index from the third information based on the weights included in the third information corresponding to the second weight matrix, the third index corresponding to the weights, and the second index.

[0018] The processor can be configured to: determine the position of the additional weight in the second weight matrix based on the second index and the third index, erase (or delete, remove or eliminate) the additional weight from the third information based on the position of the additional weight, and obtain the first weight matrix by modifying the index of the weight that is adjacent to the position of the erased additional weight in the third weight information.

[0019] According to another embodiment of this disclosure, a control method for an electronic device storing first information and second information includes: obtaining a first weight matrix by loading the first information, or obtaining a second weight matrix by loading the first information and the second information, based on resource information of the electronic device. The first information includes weights associated with the first weight matrix and first indices corresponding to the weights, and the second information includes additional weights for obtaining the second weight matrix and second indices corresponding to the additional weights.

[0020] The control method further includes: obtaining output data by inputting input data into an artificial intelligence model using a first weight matrix or a second weight matrix, and obtaining a second weight matrix based on the first weight matrix or a first weight matrix based on the second weight matrix, based on changes to the resource information of the electronic device.

[0021] The second weight matrix can be obtained based on the weights and first index included in the first information, the additional weights, and the second index.

[0022] In obtaining the second weight matrix, the additional weights can be combined with the weights included in the first information based on the first and second indices.

[0023] In obtaining the second weight matrix, the position of the weight corresponding to the first index in the second weight matrix can be determined based on the first index, and the position of the additional weight corresponding to the second index in the second weight matrix can be determined based on the second index. Based on the position of the weight corresponding to the first index and the position of the weight corresponding to the second index, at least one of the first index or the second index can be changed, and the second weight matrix can be obtained based on the position of the weight, the position of the additional weight, and the changed index.

[0024] Resource information may include at least one of the following: the available capacity of the electronic device's internal memory, the power status of the electronic device, the resources of an application running in the electronic device, or the resources of an artificial intelligence model running in the electronic device.

[0025] The first information may include a second threshold or a greater weight in the information obtained by pruning the weights less than a first threshold in the weight information, and an index corresponding to the weights less than the second threshold or a greater weight. The second information may include weights less than a second threshold in the information obtained by pruning, and an index corresponding to the weights less than the second threshold.

[0026] The control method may further include modifying the first information by performing first training based on the first information, modifying the second information by performing second training based on the first information and the second information, and obtaining a first weight matrix by loading the modified first information or obtaining a second weight matrix by loading the modified first information and the modified second information.

[0027] The control method may further include obtaining the first weight matrix by erasing additional weights corresponding to the second index from the third information based on the weights included in the third information corresponding to the second weight matrix, the third index corresponding to the weights, and the second index.

[0028] In obtaining the first weight matrix, the position of the additional weight in the second weight matrix can be determined based on the second and third indices. The additional weight can be erased from the third information based on the position of the additional weight. The first weight matrix can be obtained by modifying the index of the weight that is adjacent to the position of the erased additional weight in the third weight information.

[0029] Before proceeding with the detailed description below, it may be advantageous to define certain words and phrases used throughout this patent document: the terms “comprising” and “including” and their derivatives mean including but not limited to; the term “or” is inclusive, meaning and / or; the phrases “associated with” and “associated with” and their derivatives may mean including, being included in, interconnected, containing, contained within, connected to or connected with, coupled to or coupled with, communicable to, cooperating with, interleaved, juxtaposed, proximate, bound to or bound with, having, possessing the properties of, etc.; and the term “controller” means any device, system, or part thereof that controls at least one operation, such device may be implemented in hardware, firmware, or software, or some combination of at least two of hardware, firmware, or software. It should be noted that the functionality associated with any particular controller can be centralized or distributed, whether local or remote.

[0030] Furthermore, the various functions described below can be implemented or supported by one or more computer programs, each computer program being formed by computer-readable program code and embodied in a computer-readable medium. The terms "application" and "program" refer to one or more computer programs, software components, instruction sets, procedures, functions, objects, classes, instances, associated data, or portions suitable for implementation in a suitable computer-readable program. The phrase "computer-readable program code" includes any type of computer code, including source code, object code, and executable code. The phrase "computer-readable medium" includes any type of medium that can be accessed by a computer, such as read-only memory (ROM), random access memory (RAM), hard disk drive, optical disc (CD), digital video disc (DVD), or any other type of storage. "Non-transitory" computer-readable medium excludes wired, wireless, optical, or other communication links that transport transient electrical or other signals. Non-transitory computer-readable medium includes media that can permanently store data and media such as rewritable optical discs or erasable memory devices that can store data and subsequently rewrite it.

[0031] Throughout this patent document, definitions of certain words and phrases are provided, and those skilled in the art should understand that, in many (if not most) cases, such definitions apply to the prior and future use of the words and phrases defined in this way. Attached Figure Description

[0032] The above and / or other aspects of this disclosure will become clearer by referring to the accompanying drawings and description of certain embodiments therein, wherein:

[0033] Figure 1 This is a block diagram illustrating an electronic device according to an embodiment of the present disclosure;

[0034] Figure 2 It is used to describe Figure 1 A block diagram detailing the configuration of the electronic device;

[0035] Figure 3 This is a diagram used to illustrate zero-skipping operation according to embodiments of the present disclosure;

[0036] Figure 4 This is a diagram illustrating a zero-jump operation according to another embodiment of the present disclosure;

[0037] Figure 5A It is used to describe based on Figure 4 A diagram illustrating an example operation of using information generated from the zero-jump operation of an embodiment to obtain a weight matrix;

[0038] Figure 5B This is a diagram illustrating another example of the operation used to obtain the weight matrix;

[0039] Figure 6A These are diagrams illustrating various examples of obtaining the weight matrix;

[0040] Figure 6B This is a diagram used to illustrate other examples of obtaining the weight matrix;

[0041] Figure 7 This is a diagram used to describe pruning and zero-jump operations;

[0042] Figure 8 This is a diagram illustrating an example of obtaining weight information;

[0043] Figure 9 This is a diagram illustrating an example of generating weight information;

[0044] Figure 10A This is a diagram illustrating another example of how weight information is generated;

[0045] Figure 10B It is used to describe in Figure 10A A diagram illustrating the application of another calculation procedure in the example;

[0046] Figure 10C This is another example of a diagram used to describe the generation of weight information;

[0047] Figure 11 It is a diagram used to describe the distribution of weights changed based on pruning operations;

[0048] Figure 12 This is a diagram illustrating an example of resource-based weighting without considering electronic devices;

[0049] Figure 13 This is a diagram illustrating an example of determining weights based on the resources of electronic devices;

[0050] Figure 14A It is a diagram used to describe the operation of classifying multiple weights into multiple classes;

[0051] Figure 14B It is a diagram used to describe the computational process of classifying multiple weights into multiple classes;

[0052] Figure 14C This is for further description. Figure 14B A diagram illustrating the operation;

[0053] Figure 14D It is a diagram used to describe the distribution of weights;

[0054] Figure 15 This is a diagram illustrating an example of performing artificial intelligence analysis based on weight information that has been classified into multiple classes;

[0055] Figure 16 This is a diagram illustrating another example of performing artificial intelligence analysis based on weight information that has been classified into multiple classes;

[0056] Figure 17 This is a diagram used to describe an example of identifying classes by considering resource information;

[0057] Figure 18 This is a diagram used to describe another example of identifying classes by considering resource information;

[0058] Figure 19 This is a diagram illustrating yet another example of using resource information to identify classes;

[0059] Figure 20 It is a flowchart used to describe the operation of setting operation points;

[0060] Figure 21 These are diagrams illustrating a control method for an electronic device according to embodiments of the present disclosure; and

[0061] Figure 22 This is a diagram illustrating a comparison of delay changes based on a change in the operating point, according to an embodiment of the present disclosure. Detailed Implementation

[0062] The following discussion Figures 1 to 22 The various embodiments used to describe the principles of this disclosure in this patent document are for illustrative purposes only and should not be construed as limiting the scope of this disclosure in any way. Those skilled in the art will understand that the principles of this disclosure can be implemented in any suitably arranged system or device.

[0063] This disclosure will be described in detail below with reference to the accompanying drawings.

[0064] Considering the function of this disclosure, currently widely used general terms are selected as the terms used in the embodiments of this disclosure, but these may be changed depending on the intent of those skilled in the art, judicial precedents, the emergence of new technologies, etc. Furthermore, in certain circumstances, terms arbitrarily chosen by the applicant may exist. In such cases, the meaning of such terms will be mentioned in detail in the corresponding descriptive section of this disclosure. Therefore, the terms used in this disclosure should be defined based on their meaning and the content throughout this disclosure, rather than simply their names.

[0065] In this disclosure, expressions such as “having,” “may have,” “including,” and “may include” indicate the presence of corresponding features (e.g., numerical values, functions, operations, components such as parts), and do not exclude the presence of additional features.

[0066] The expression “at least one of A and / or B” should be understood to mean either “A” or “B” or “A and B”.

[0067] The terms “first,” “second,” etc., used in this disclosure may refer to various components regardless of the order and / or importance of the components, and will only be used to distinguish one component from another, without limiting the corresponding components.

[0068] When it is mentioned that any component (e.g., the first component) is (operably or communicatively) connected to or coupled to another component (e.g., the second component) / coupled to another component, it will be understood that any component is directly coupled to / coupled to another component, or can be coupled to / coupled to another component through another component (e.g., the third component).

[0069] Unless the context clearly indicates otherwise, singular expressions include plural expressions. It should also be understood that the terms "comprising" or "constituting" as used herein specify the presence of features, numbers, steps, operations, components, parts, or combinations thereof mentioned in the specification, but do not exclude the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0070] In this disclosure, a "module" or "device" can perform at least one function or operation and can be implemented by hardware or software or by a combination of hardware and software. Furthermore, multiple "modules" or multiple "devices" can be integrated into at least one module and implemented as at least one processor (not shown) in addition to the "module" or "device," which requires implementation by specific hardware.

[0071] In this disclosure, the term "user" can refer to a person using an electronic device or a device using an electronic device (e.g., an artificial intelligence electronic device).

[0072] In the following, embodiments of the present disclosure will be described in more detail with reference to the accompanying drawings.

[0073] The AI-related functions according to this disclosure operate via a processor and memory. The processor can be configured as one or more processors. In this case, the one or more processors can be general-purpose processors such as CPUs, APs, or digital signal processors (DSPs), graphics-specific processors such as GPUs or vision processing units (VPUs), or AI-specific processors such as neural network processing units (NPUs). The one or more processors perform control to process input data according to predefined operating rules or AI models stored in memory. Alternatively, when the one or more processors are AI-specific processors, the AI-specific processors can be designed in a hardware architecture dedicated to processing a specific AI model.

[0074] The characteristic of predefined operating rules or artificial intelligence models is that they are created through training (learning). Here, creation through training means creating a set of predefined operating rules or artificial intelligence models to perform desired characteristics (or purposes) by training a basic artificial intelligence model using multiple training data sets by a training algorithm. Such training can be performed on the device itself that performs the artificial intelligence according to this disclosure, or it can be performed via a separate server and / or system. Examples of training algorithms include supervised training (learning), unsupervised training (learning), semi-supervised training (learning), or reinforcement training (learning), but training algorithms are not limited to the examples mentioned above.

[0075] Artificial intelligence models can include multiple neural network layers. Each of the multiple neural network layers has multiple weight values ​​and performs neural network operations through operations between the results of operations in the previous layer and the multiple weights. The multiple weights of the multiple neural network layers can be optimized through the training results of the artificial intelligence model. For example, multiple weights can be updated to reduce or minimize the loss or cost values ​​obtained from the artificial intelligence model during the training process. Artificial neural networks can include deep neural networks (DNNs), examples of which include, but are not limited to, convolutional neural networks (CNNs), deep neural networks (DNNs), recurrent neural networks (RNNs), restricted Boltzmann machines (RBMs), deep belief networks (DBNs), bidirectional recurrent deep neural networks (BRDNNs), or deep Q-networks.

[0076] Figure 1 This is a block diagram illustrating an electronic device according to an embodiment of the present disclosure.

[0077] Reference Figure 1 The electronic device 100 may include a memory 110 and a processor 120.

[0078] Electronic device 100 according to various embodiments of this disclosure may include at least one of, for example, a smartphone, tablet PC, mobile phone, webcam phone, desktop PC, laptop PC, netbook computer, workstation, server, personal digital assistant (PDA), portable multimedia player (PMP), MP3 player, camera, or wearable device. Wearable devices may include at least one of accessory-type wearable devices (e.g., watches, rings, bracelets, anklets, necklaces, glasses, contact lenses, or head-mounted devices (HMDs)), textile or clothing-integrated wearable devices (e.g., electronic clothing), body accessory-type wearable devices (e.g., skin pads or tattoos), or bio-implantable circuitry. In some embodiments, electronic device 100 may include at least one of, for example, a television, digital video disc (DVD) player, or audio player.

[0079] Memory 110 can be implemented as an internal memory such as ROM (e.g., electrically erasable programmable read-only memory (EEPROM)) or RAM included in processor 120, or as a separate memory from processor 120. In this case, depending on the data storage purpose, memory 110 can be implemented as a memory embedded in electronic device 100 or as a memory that can be attached to and removable from electronic device 100. For example, data for driving electronic device 100 can be stored in memory embedded in electronic device 100, and data for extended functions of electronic device 100 can be stored in memory that can be attached to and removable from electronic device 100.

[0080] Meanwhile, the memory embedded in the electronic device 100 can be implemented as volatile memory (e.g., dynamic random access memory (DRAM), static RAM (SRAM), synchronous dynamic RAM (SDRAM), etc.), or non-volatile memory (e.g., one-time programmable read-only memory (OTPROM), programmable ROM (PROM), erasable and programmable ROM (EPROM), electrically erasable and programmable ROM (EEPROM), mask ROM, flash ROM, flash memory (e.g., NAND flash, NOR flash, etc.), hard disk drive or solid-state drive (SSD)), and can be implemented in the form of at least one of the following: memory that can be attached to the electronic device 100 and memory that can be removed from the electronic device 100, such as memory cards (e.g., compact flash (CF), secure digital (SD), micro-secure digital (Micro-SD), mini-secure digital (Mini-SD), extreme digital (xD), multimedia card (MMC), etc.), external memory that can be connected to a USB port (e.g., USB memory).

[0081] The memory 110 can store at least one instruction. Here, an instruction can refer to at least one of a user command, a user operation, or a preset event.

[0082] The memory 110 according to embodiments of this disclosure can store artificial intelligence models. However, the artificial intelligence models are not necessarily stored in the memory 110 of the electronic device 100, and can be implemented by storing them on an external server.

[0083] The processor 120 can perform overall control operations of the electronic device 100.

[0084] Processor 120 can be implemented as a digital signal processor (DSP), a microprocessor, or a time controller (TCON) for processing digital signals. However, processor 120 is not limited to these, and may include one or more of a central processing unit (CPU), microcontroller unit (MCU), microprocessor unit (MPU), controller, application processor (AP), graphics processing unit (GPU), communication processor (CP), and ARM processor, or can be defined by the corresponding terms. Furthermore, processor 120 can also be implemented as a system-on-a-chip (SoC) or a large-scale integration (LSI) of embedded processing algorithms, and may also be implemented in the form of a field-programmable gate array (FPGA). In addition, processor 120 can perform various functions by executing computer-executable instructions stored in memory 110.

[0085] The processor 120 may include resource information based on the electronic device 100, and may obtain a first weight matrix by loading first information or a second weight matrix by loading first and second information. Here, the first information may include weights associated with the first weight matrix and a first index corresponding to the weights associated with the first weight matrix, and the second information may include additional weights for obtaining the second weight matrix and a second index corresponding to the additional weights.

[0086] The processor 120 can determine, based on resource information, whether to load the first information or both the first and second information. Here, the first and second information can refer to information related to the weights corresponding to the first and second classes.

[0087] Here, a class can be a group of weight values ​​obtained during the compression process of the weight matrix used in the artificial intelligence model, categorized by size. For example, suppose the original weight matrix contains 100 weights. Suppose that as the original weight matrix is ​​compressed (e.g., pruned), 50 of the 100 weights are pruned to 0, and 50 weights have non-zero values. Then, weights with a threshold value or greater among the 50 weights can be assigned to the first class, and weights with a value less than the threshold among the 50 weights can be assigned to the second class. Meanwhile, the number of classes, the number of thresholds, and the threshold values ​​can vary depending on the implementation.

[0088] Therefore, in this disclosure, obtaining a first weight matrix by loading first information can refer to obtaining a weight matrix based on information about the weights included (or assigned) in the first class. Furthermore, obtaining a second weight matrix by loading first information and second information can refer to obtaining a weight matrix based on information about both the weights included in the first class and the weights included in the second class.

[0089] Simultaneously, the processor 120 can determine, based on the resource information of the electronic device 100, whether to use weight information corresponding to the first type (or weight data, hereinafter collectively referred to as weight information for convenience) or weight information corresponding to the first and second types. Here, resource information can refer to information related to resources associated with the hardware of the current electronic device 100, resources required for executing a specific module or application, etc. For example, resource information may include at least one of the following: the internal memory capacity of the electronic device 100 (total capacity, used capacity, available capacity), the power state of the electronic device 100, the resources of the application executing in the electronic device 100, or the resources of the artificial intelligence model executing in the electronic device 100.

[0090] Here, the used capacity of internal memory can refer to the capacity of internal memory that has been used out of the total capacity, and the available capacity can refer to the capacity of internal memory that is currently available. For example, the available capacity can be calculated by subtracting the used capacity of internal memory from the total capacity of internal memory.

[0091] The power state of electronic device 100 can refer to the current power state of electronic device 100 or the currently set power mode. For example, if the power of electronic device 100 is currently insufficient and both the first and second information are uploaded, the load may be high. Therefore, processor 120 can obtain the weight matrix by loading only the first information. Alternatively, when electronic device 100 is in a low-power mode, the weight matrix can be obtained by loading only the first information.

[0092] The resources of an application running in electronic device 100 may be related to the accuracy of the weight matrix required by the artificial intelligence model used in the currently running application. For example, if the weight matrix used in the currently running application is a first weight matrix that can be obtained using only the first information, then only the first information can be loaded; and if the weight matrix used in the currently running application is a second weight matrix that can be obtained using both the first and second information, then both the first and second information can be loaded.

[0093] Furthermore, even when using the same application or the same AI model, the required resources can change in real time. For example, the weight matrix required to perform the first function in the same application may differ from the weight matrix required to perform the second function. See later for more details. Figure 12 and 13 Describe the specific operations.

[0094] As described above, the processor 120 can determine whether to use the weight information corresponding to the first type, i.e., the first information, or whether to use the weight information corresponding to the first type and the second type, i.e., both the first information and the second information, based on the resource information of the electronic device 100.

[0095] As described above, processor 120 can identify class combinations based on resource information and obtain weight information corresponding to the identified class combinations. Then, processor 120 can obtain a weight matrix based on the obtained weight information. Then, processor 120 can obtain output data based on the obtained weight matrix and input data.

[0096] Simultaneously, resource information can change in real time, and processor 120 can obtain resource information of electronic device 100 in real time. Then, if processor 120 identifies, based on the obtained resource information, that the resources have changed by a threshold or more, processor 120 can obtain a new weight matrix. Here, the threshold can be obtained based on the available resources of internal memory and the resources of each class.

[0097] Examples of changing resource information can include increasing or decreasing the amount of currently used resources. Since decreasing currently used resources increases available resources, the operation proceeds without issue. However, if increasing currently used resources decreases available resources, problems may arise.

[0098] For ease of description, the following description will assume that the resource information of electronic device 100 is related to the available capacity of the internal memory of processor 120.

[0099] Processor 120 can determine the available capacity of internal memory in real time. When the available capacity of internal memory decreases while using a second weight matrix obtained by currently using the first and second information, processor 120 can obtain a first weight matrix by excluding the second information from the second weight matrix, thereby changing the second weight matrix into a first weight matrix. On the other hand, there may be situations where the number of targets for artificial intelligence analysis increases or a relatively high level of accuracy is required. When the number of targets for artificial intelligence analysis increases while using a first weight matrix obtained by currently using the first information, processor 120 can obtain a second weight matrix by adding the second information to the first weight matrix, thereby changing the first weight matrix into a second weight matrix. Specific methods for obtaining the first weight matrix or the second weight matrix using the first information, or the first and second information, will be described below.

[0100] If a first weight matrix is ​​required for an artificial intelligence model, the processor 120 can obtain first weight information (or first information) corresponding to the first class. Furthermore, the first weight information may include the weights included in the first class and a first index corresponding to the weights included in the first class. Here, the first index may include multiple indices corresponding to each of the multiple weights. For example, if the number of weights included in the first class is three, the number of first indices may also be three.

[0101] According to an embodiment, the processor 120 can obtain a first weight matrix based on first weight information. Here, the first weight matrix is ​​obtained using only the first type and can be a matrix reflecting relatively high weights.

[0102] If the second weight matrix is ​​to be applied to an artificial intelligence model, the processor 120 can obtain second weight information (or second information) corresponding to the second class and first weight information (or first information) corresponding to the first class. The second weight information may include the weights included in the second class and a second index corresponding to the weights included in the second class. Here, since the weights included in the second class are used together with the weights included in the first class to construct the second weight matrix, the weights included in the second class will be referred to as additional weights below.

[0103] Meanwhile, the processor 120 can obtain a second weight matrix by combining the additional weights with the weights included in the first weight information based on the first index and the second index.

[0104] Specifically, the processor 120 can determine the position of the weight corresponding to the first index in the second weight matrix based on the first index, determine the position of the additional weight corresponding to the second index in the second weight matrix based on the second index, change at least one of the first index or the second index based on the position of the weight corresponding to the first index and the position of the weight corresponding to the second index, and obtain the second weight matrix based on the position of the weight, the position of the additional weight and the changed index.

[0105] When generating the second weight matrix, the processor 120 can identify the position of the weights included in the first weight information based on the first index included in the first weight information, and can identify the position of the weights (additional weights) included in the second weight information based on the second index included in the second weight information.

[0106] Here, when generating the second weight matrix, the processor 120 can change at least one of the first or second indices as needed. Because the second weight matrix is ​​a combination of the first and second weight information, the first and second indices can be used differently. Therefore, the values ​​of either the first or second index can be changed when generating the second weight matrix. However, the initially stored initial index (one of the first or second index) may not be changed. For example, in... Figure 5A In the original text, the first index in the first weight information is {3, 10}, and the second index in the second weight information is {9}. However, the index of the newly created weight information is {3, 5, 4}. Only the initial index 3 is retained, and the values ​​of both the first and second indices can be changed.

[0107] Reference Figures 5A to 9 This describes the detailed operations for generating new weight information and a new weight matrix based on combination operations.

[0108] Meanwhile, the first weight information may include a second threshold or greater weight in the information obtained by pruning weights less than the first threshold and an index corresponding to the second threshold or greater weight, and the second weight information may include weights less than the second threshold in the information obtained by pruning and an index corresponding to the weights less than the second threshold.

[0109] Here, weight information can refer to a set of raw weights that are not classified according to class. Weight information can correspond to... Figure 7 The attached figure is labeled 706.

[0110] Here, the pruning operation can refer to the operation of changing weights less than a threshold to zero. The processor 120 can perform a pruning operation that changes weights less than a first threshold among multiple weights included in the weight information to zero.

[0111] The weight information that changes by performing a pruning operation can be described as the pruning weight information. The pruning weight information can correspond to... Figure 7 The attached figure is labeled 711.

[0112] Processor 120 can classify multiple weights included in the pruned weight information into multiple classes. Specifically, processor 120 can classify multiple weights included in the pruned weight information into multiple classes based on the magnitude of the weights. Processor 120 can assign weights with a magnitude value (or importance) of a second threshold or greater among the multiple weights included in the pruned weight information to the first class, and weights with a magnitude value less than the second threshold to the second class. Weights with relatively high magnitude values ​​can be assigned to the first class, while weights with relatively low magnitude values ​​can be assigned to the second class. Here, the first threshold and the second threshold can be used as different criteria and can have different values.

[0113] Simultaneously, the processor 120 can obtain the weights included in the first class (class a11) among the multiple weights included in the pruned weight information (class A1), and a first index corresponding to the weights included in the first class (class a12). Furthermore, the processor 120 can obtain the weights included in the second class (class a12) among the multiple weights included in the pruned weight information (class A1), and a second index corresponding to the weights included in the second class (class a12).

[0114] Simultaneously, the first information may include weights (class a13) obtained by performing first training on weights (class a11) of a second threshold or greater in the information obtained through pruning (class A1) (pruned weight information) and an index corresponding to the weights (class a13) obtained through the first training. The second information may include weights (class a14) obtained by performing second training on weights (class a13) of a second threshold or greater and weights (class a12) of a second threshold or less in the information obtained through pruning (class A1) (pruned weight information) and an index corresponding to the weights (class a14) obtained through the second training. The weights (class a13) of the second threshold or greater may be fixed during the second training. Furthermore, the processor 120 may be configured to change the first information (class a11) by performing first training based on the first information, change the second information (class a12) by performing second training based on the first information and the second information, and obtain a first weight matrix by loading the changed first information (class a13), or obtain a second weight matrix by loading the changed first information (class a13) and the changed second information (class a14). Furthermore, the processor 120 can be configured to obtain weights (class a13) by performing a first training on weights (class a11) of a second threshold or greater in the information obtained through pruning (class A1), obtain an index corresponding to the weights (class a13) obtained through the first training, change the weights (class a11) and indexes included in the first information to the weights (class a13) obtained through the first training and the indexes corresponding to the weights (class a13) obtained through the first training, and obtain weights (class a14) of a second training on the weights (class a13) obtained through the first training and weights (class a14) of a second training that are less than the second threshold (class a14) and weights (class a14) of a second training that are less than the second threshold in the information obtained through pruning. The weights of the threshold (class a12) are obtained, and the indices corresponding to the weights (class a14) less than the second threshold among the weights (classes a13 and a14) obtained through the second training are obtained. The weights (class a12) and indices included in the second information are changed to the weights (class a14) less than the second threshold among the weights (classes a13 and a14) obtained through the second training and the indices corresponding to the weights (classes a14) less than the second threshold among the weights (classes a13 and a14) obtained through the second training. A first weight matrix is ​​obtained by loading the changed first information (class a13), or a second weight matrix is ​​obtained by loading the changed first information (class a13) and the changed second information (class a14). Furthermore, during the second training, the weights (class a13) of the second threshold or greater are fixed.

[0115] Here, the first training and the second training can correspond to the retraining process. The retraining process can refer to the process of checking the modified weight information to reduce errors. The existing weight information may change during the retraining process. The first training can be retraining for the first class, and the second training can be retraining for multiple classes.

[0116] Referring to step S1421 of Figure 14, the weights included in a specific class can be changed during retraining. The processor 120 can assign weights from the pruned weight information, with a second threshold or greater, to the first class. Furthermore, the processor 120 can retrain (initially train) the weights assigned to (included in) the first class. Here, the weights included in the first class can be changed during retraining (initially train).

[0117] Reference Figure 14A In step S1426, when retraining multiple classes including weights, the weights can remain unchanged for some classes and can be changed for the remaining classes. The processor 120 can assign weights from the pruned weight information that are at or above a second threshold to the first class, and can assign weights from the pruned weight information that are below the second threshold to the second class. Furthermore, the processor 120 can retrain (second training) the weights assigned to (included in) the first and second classes. Here, the weights included in the first class can be fixed during retraining (second training), while only the weights included in the second class can be changed during retraining (second training). When performing retraining on multiple classes, the weights included in classes with relatively high weights can be fixed, and the weights included in classes with relatively low weights can be changed.

[0118] Simultaneously, the processor 120 can perform pruning operations based on weight information, perform zero-jump operations on the pruned weight information, and classify the weight information that has undergone pruning and zero-jump operations into multiple classes. (See below for further details.) Figure 3 and Figure 7 The zero-jump operation is described. Furthermore, according to one embodiment, at least one of the following operations—the pruning operation, the zero-jump operation, or the operation of classifying weight information into multiple classes—may be omitted, and their order may also be changed.

[0119] Meanwhile, the first weight matrix can be a matrix obtained using the first weight information, and the second weight matrix can be a matrix obtained using both the first and second weight information. Specifically, the processor 120 can obtain the third weight information by combining the first and second weight information to generate the second weight matrix. The third weight information can include the weights included in the first and second classes, as well as the indices (third indices) corresponding to the weights included in the first and second classes. Furthermore, the processor 120 can obtain the second weight matrix based on the obtained third weight information.

[0120] Processor 120 can perform an operation to erase weights included in a specific class from existing weight information. Assume processor 120 has already obtained a second weight matrix using first and second weight information. During artificial intelligence analysis using the second weight matrix, there may be a situation where insufficient internal memory leads to resource shortages. Here, processor 120 can modify the weight information or weight matrix to reduce the resources required for artificial intelligence analysis. For example, processor 120 can perform an operation to erase weights included in classes with low weights among multiple classes to reduce resources.

[0121] The processor 120 can obtain the first weight matrix by erasing the additional weight corresponding to the second index from the third weight information, based on the weights included in the third weight information corresponding to the second weight matrix, the third index corresponding to the weights, and the second index.

[0122] Here, the third weight information corresponding to the second weight matrix can be information obtained by combining the first and second weight information. Furthermore, the weights included in the third weight information can refer to the weights included in the first and second categories. Additionally, the third index corresponding to this weight can refer to the index corresponding to the weights included in the first and second categories. Furthermore, the additional weight corresponding to the second index can refer to the weights included in the second category.

[0123] Here, the processor 120 can determine the position of the additional weight in the second weight matrix based on the second index and the third index, erase the additional weight from the third weight information based on the position of the additional weight, and obtain the first weight matrix by modifying the index of the weight adjacent to the position of the erased additional weight in the third weight information.

[0124] Here, additional weights can refer to the weights included in the second category.

[0125] Processor 120 can perform an operation to erase the weights of a specific class among multiple classes based on a preset event. Specifically, processor 120 can erase the weights included in the second class among the weights (including weights in the first and second classes) in the third weight information (or second weight matrix) obtained based on the resource information of electronic device 100. Processor 120 can use a second index to erase only the weights included in the second class among multiple weights. Because the position of the weights included in the second class can be detected as long as the second index is known, even if the magnitude value (or importance) of the weights included in the second class is unknown, only the weights included in the second class can be erased. Therefore, processor 120 can erase the weights included in the second class among the weights (including weights in the first and second classes) in the obtained third weight information (or second weight matrix) based on the third weight information and the second index. In addition, processor 120 can change the index of the weights that are immediately adjacent to the weights to be erased in the third index. Specifically, if there are two adjacent indices, processor 120 can obtain a new index by summing the data of each index in the two indices. See below for further details. Figure 10A , Figure 10B and Figure 10C Describe the detailed calculation process.

[0126] Specifically, the processor 120 can obtain new weight information by erasing the weights included in the second category from the weights included in the third weight information (or the second weight matrix) (including weights in the first and second categories). Here, the new weight information can correspond to the first weight information.

[0127] The processor 120 can obtain output data based on the acquired weight matrix and input data. Specifically, the processor 120 can perform multiplication operations using the acquired weight matrix and input data, and obtain output data as the result of the operation.

[0128] According to this disclosure, by enabling scalable operating scenarios for various neural networks, user satisfaction can be maximized and the competitiveness of installed products can be improved. Furthermore, according to this disclosure, because the operating point can be selected based on power consumption, user satisfaction with electronic devices (e.g., mobile devices) with low memory capacity can be maximized.

[0129] Specifically, according to this disclosure, class combinations can be determined by classifying weight information into multiple classes based on the resource information of the electronic device. Therefore, even in situations with insufficient resources, artificial intelligence analysis can be performed by appropriately selecting class combinations or generating new weight matrices.

[0130] Furthermore, this disclosure has described that the format in which the weights of the electronic device 100 are stored is run-length encoded (RLC). However, according to embodiments, the format in which the weights are stored may be Compressed Sparse Rows (CSR) or Compressed Sparse Columns (CSC). CSR is a method of rearranging rows in a horizontal order within a matrix, and may refer to a method of organizing and compressing rows. CSR is a method of rearranging rows in both vertical and horizontal order within a matrix, and may refer to applying the method of organizing rows to CSR columns.

[0131] Furthermore, while only a simple configuration of the electronic device 100 has been shown and described above, various other configurations may be provided in practice. This will be referred to below. Figure 2 Describe it.

[0132] Figure 2 It is used to describe Figure 1 A block diagram showing the detailed configuration of the electronic device.

[0133] Reference Figure 2 The electronic device 100 may include a memory 110, a processor 120, a communication interface 130, a display 140, a user interface 150, and an input / output interface 160.

[0134] At the same time, redundant descriptions of operations that are the same as those described above in the operation of memory 110 and processor 120 will be omitted.

[0135] Processor 120 typically controls the operation of electronic device 100 using various programs stored in memory 110. Specifically, processor 120 includes random access memory (RAM), read-only memory (ROM), a main central processing unit (CPU), first to nth interfaces, and a bus. RAM, ROM, main CPU, first to nth interfaces, etc., can be interconnected via the bus. The instruction set used for booting the system, etc., is stored in ROM. When a power-on command is input to supply power to the main CPU, the main CPU copies the operating system (O / S) stored in memory 110 to RAM according to the instructions stored in ROM, and executes the O / S to boot the system. When booting is complete, the main CPU copies various application programs stored in memory 110 to RAM, and executes the copied application programs to perform various operations. The main CPU accesses memory 110 to perform booting using the O / S stored in memory 110. Furthermore, the main CPU performs various operations using various programs, contents, data, etc., stored in memory 110. First to nth interfaces are connected to the aforementioned components. One of the interfaces can also be a network interface connected to an external device via a network.

[0136] Communication interface 130 is a component that performs communication with various types of external devices according to various communication methods. Communication interface 130 includes Wi-Fi modules, Bluetooth modules, infrared communication modules, wireless communication modules, etc. Here, each communication module can be implemented in the form of at least one hardware chip.

[0137] The Wi-Fi module and Bluetooth module communicate using Wi-Fi and Bluetooth methods, respectively. When using either a Wi-Fi or Bluetooth module, various connection information such as Service Set Identifier (SSID) and session key are first sent and received to establish communication, and then various other information can be sent and received.

[0138] The infrared communication module performs communication based on Infrared Data Association (IrDA) technology, which wirelessly transmits data over short distances using infrared light, which falls between visible light and millimeter waves.

[0139] The wireless communication module may include at least one communication chip that performs communication according to various wireless communication protocols, such as Zigbee, 3G, 3GPP, LTE, LTE-A Advanced, 4G, 5G, etc., in addition to the communication methods mentioned above.

[0140] In addition, the communication interface 130 may include at least one of a local area network (LAN) module, an Ethernet module, or a wired communication module for performing communication using a pair cable, a coaxial cable, or a fiber optic cable.

[0141] According to one example, communication interface 130 can use the same communication module (e.g., a Wi-Fi module) to communicate with external devices such as remote controllers and external servers.

[0142] According to another example, communication interface 130 can use different communication modules (e.g., Wi-Fi modules) to communicate with external devices such as remote controllers and external servers. For example, communication interface 130 can use at least one of an Ethernet module or a Wi-Fi module to communicate with an external server, and can use a BT module to communicate with external devices such as remote controllers. However, this is only an example, and when communicating with multiple external devices or external servers, communication interface 130 can use at least one of a variety of communication modules.

[0143] The display 140 can be implemented as various types of displays, such as liquid crystal displays (LCDs), organic light-emitting diode (OLED) displays, plasma display panels (PDPs), etc. The display 140 may also include driving circuitry, backlight units, etc., which can be implemented in the form of amorphous silicon thin-film transistors (TFTs), low-temperature polycrystalline silicon (LTPS) TFTs, and organic TFTs (OTFTs). Furthermore, the display 140 can be implemented as a touchscreen combined with touch sensors, flexible displays, three-dimensional (3D) displays, etc.

[0144] Furthermore, according to embodiments of this disclosure, the display 140 may include a bezel accommodating a display panel and a display panel for outputting images. Specifically, according to embodiments of this disclosure, the bezel may include a touch sensor (not shown) for sensing user interaction.

[0145] User interface 150 can be implemented as a device such as a button, touchpad, mouse, or keyboard, or it can also be implemented as a touchscreen that can perform the aforementioned display and manipulation / input functions. Here, the button can be various types of buttons such as mechanical buttons, touchpads, scroll wheels, etc., and they are formed in any area, such as the front, side, and rear of the main body of the electronic device 100.

[0146] The input / output interface 160 can be any of the following interfaces: High Definition Multimedia Interface (HDMI), Mobile High Definition Link (MHL), Universal Serial Bus (USB), DisplayPort (DP), Thunderbolt, Video Graphics Array (VGA) port, RGB port, D-SUB, and Digital Vision Interface (DVI).

[0147] The input / output interface 160 can input and output at least one of audio and video signals.

[0148] According to the implementation example, the input / output interface 160 may include a port for inputting and outputting audio signals only and a port for inputting and outputting video signals only, which are separate ports, or it may be implemented as a single port for inputting and outputting both audio signals and video signals.

[0149] Electronic device 100 may include a speaker (not shown). The speaker (not shown) may be a component that not only outputs various audio data processed by input / output interface 160, but also outputs various notification sounds or voice messages.

[0150] The electronic device 100 may also include a microphone (not shown). A microphone is a component used to receive user voice or other sounds and convert user voice or other sounds into audio data.

[0151] A microphone (not shown) can receive user voice when activated. For example, the microphone can be integrated on the upper side of the electronic device 100, or in the front, side, or other directions. The microphone may include various components, such as a microphone for collecting user voice in analog form, an amplifier circuit for amplifying the collected user voice, an A / D conversion circuit for sampling and converting the amplified user voice into a digital signal, and a filter circuit for removing noise from the converted digital signal.

[0152] A camera (not shown) is a component used to generate captured images by capturing a subject, where the concept of captured images includes both moving and still images.

[0153] A camera (not shown) can acquire images from at least one external device and can be implemented as a camera, lens, infrared sensor, etc.

[0154] The camera (not shown) may include a lens and an image sensor. The type of lens includes general-purpose lenses, wide-angle lenses, and zoom lenses, and can be determined based on the type, characteristics, and usage environment of the electronic device 100. As the image sensor, complementary metal-oxide-semiconductor (CMOS) and charge-coupled device (CCD) devices may be used.

[0155] A camera (not shown) outputs incident light as an image signal. Specifically, the camera (not shown) may include a lens, pixels, and an analog-to-digital converter (AD converter). The lens can collect light from a subject and form an optical image in the capture area, and the pixels can output the light incident through the lens as an analog image signal. Furthermore, the AD converter can convert the analog image signal into a digital image signal and output the converted digital image signal. In particular, the camera (not shown) is positioned in front of the capturing electronic device 100 and can generate a captured image by capturing a user present in front of the electronic device 100.

[0156] Furthermore, while describing the electronic device 100 according to embodiments of this disclosure, only one camera (not shown) is described; however, in actual implementations, multiple cameras may be installed. The electronic device 100 may include multiple cameras, and the user's head posture can be identified by combining images received from the multiple cameras. Because three-dimensional motion can be analyzed more accurately, using multiple cameras instead of a single camera allows for more effective identification of the user's head posture.

[0157] Figure 3 This is a diagram illustrating zero-jump operation according to embodiments of the present disclosure.

[0158] Reference Figure 3According to an embodiment, the electronic device 100 can process input data using the general method 305, or it can process input data based on the zero-hop method 310.

[0159] The general method 305 can be a method of multiplying the weight information 306 and the input data 307. The general method 305 can be used without applying any modification to the weight information 306, and can be a method that uses the obtained weights as is. The weight information 306 can be classified into data with a weight of 0 and data with a weight not equal to 0. Data with a weight of 0 can be classified into multiple groups, such as groups 306-1, 306-2, and 306-3, which can be classified based on consecutive 0 data. Data with a weight not equal to 0 in the weight information can be {0.7, -0.1, 0.86}. Furthermore, the electronic device can multiply the weight information 306 and the input data 307 separately according to the general method 305. Here, the electronic device 100 can perform a total of 15 multiplication calculations.

[0160] Simultaneously, the zero-jump method 310 can be a method for data with a weight of 0 in the jump weight information 306. Here, the jump operation can refer to not performing the calculation process of multiplying by zero. For example, the zero-jump method 310 can use the number of zeros as an index to exclude zero weight information from the weight information 306. Furthermore, the zero-jump method 310 can obtain the number of zeros included in each group 306-1, 306-2, and 306-3. The number of zeros in multiple groups 306-1, 306-2, and 306-3 can be 3, 5, and 4. In addition, information about the number of zeros can be used as an index. For example, the electronic device 100 can use information about the number of zeros in multiple groups 306-1, 306-2, and 306-3 as indices 311-1, 311-2, and 311-3. The index 311-1 of the first group 306-1 can refer to the number (three) of the jump input data (12, 2, and 9). Furthermore, the index 311-2 of the second group 306-2 can refer to the number of skipped input data (45, 6, 23, 3, and 12) (five). Furthermore, the index 311-3 of the third group 306-3 can refer to the number of skipped input data (9, 1, 45, and 6) (four). Additionally, the zero-skip method 310 can multiply the input data (1, 2, and 23) with data in the weight information 306 that have non-zero weights (0.7, -0.1, and 0.86), respectively. As a result, the electronic device 100 can obtain weight information corresponding to {3, 0.7, 5, -0.1, 4, 0.86} based on the weight information 306 through the zero-skip operation. Here, "3, 5, 4" can be indices, and "0.7, -0.1, 0.86" can be weights.

[0161] The electronic device 100 requires a total of 15 calculation processes according to the general method 305, while it requires a total of 3 calculation processes according to the zero-jump method 310, thereby simplifying the calculation process.

[0162] Figure 4 This is a diagram used to illustrate a zero-jump operation according to another embodiment of the present disclosure.

[0163] Reference Figure 4 Electronic device 100 can perform zero-hop operations based on the magnitude (or importance) of the weights. Electronic device 100 can perform zero-hop operations (401) by only obtaining weights whose magnitude (or importance) is a threshold or greater, or by only obtaining weights whose magnitude (or importance) is less than a threshold (402).

[0164] Regarding operation 401, electronic device 100 can classify weight information 405 into data with a weight of 0 and data with a weight not equal to 0 (0.7, -0.1, and 0.86). Furthermore, electronic device 100 can identify weights (e.g., 0.2) or greater (0.7 and 0.86) from the data with a weight not equal to 0 (0.7, -0.1, and 0.86). Electronic device 100 can determine that the weight (-0.1) less than the threshold (0.2) among the data with a weight not equal to 0 is equal to 0. Furthermore, electronic device 100 can classify weight information 405 into data groups 406-1 and 406-2 with a weight of 0 and data groups 407-1 and 407-2 with a weight not equal to 0. Additionally, as... Figure 3 The electronic device 100 can identify the number of zeros in each data group with a weight of 0. The number of zeros corresponding to group 406-1 is three, and the number of zeros corresponding to group 406-2 is ten. Here, the electronic device 100 can calculate the number of zeros by including a weight less than a threshold (0.2) in the number of zeros. For example, "-0.1" is not zero, but if the weight is less than 0.2, the data is determined to be 0; therefore, the number of zeros can be calculated by including "-0.1" in the number of zeros.

[0165] Regarding operation 401, electronic device 100 can use the number of zeros obtained as an index. The number of zeros corresponding to group 406-1 is three, and electronic device 100 can use the number of zeros obtained (three) as the index 411-1 corresponding to group 406-1. Furthermore, the number of zeros corresponding to group 406-2 is ten, and electronic device 100 can use the number of zeros obtained (ten) as the index 411-2 corresponding to group 406-2.

[0166] Regarding operation 401, electronic device 100 can obtain first weight information 410 based on weight information 405, a preset threshold, and obtained indices 411-1 and 411-2. The first weight information 410 may include indices 411-1 and 411-2, weights 407-1 and 407-2, and order information (in the order of 411-1, 407-1, 411-2, and 407-2).

[0167] Both weight information 405 and first weight information 410 can correspond to information related to weights. However, for ease of explanation, the original weights can be described as weight information 405, and the weights changed through a zero-jump operation can be described as first weight information 410.

[0168] Operation 401 describes the process of obtaining first weight information 410 by performing a zero-jump operation from weight information 405 when the magnitude of the weight is a threshold (0.2) or greater.

[0169] Regarding operation 402, when the magnitude of the weight is less than the threshold (0.2), the electronic device 100 can obtain the second weight information 415 by performing a zero-jump operation from the weight information 405.

[0170] Regarding operation 402, electronic device 100 can classify weight information 405 into data with a weight of 0 and data with a weight of non-zero (0.7, -0.1, 0.86). Furthermore, electronic device 100 can identify weights (e.g., -0.1) less than a threshold (e.g., 0.2) from the data with non-zero weights (0.7, -0.1, 0.86). Electronic device 100 can determine that weights (0.7 and 0.86) of the threshold (0.2) or greater among the data with non-zero weights are equal to 0. Here, electronic device 100 may choose not to determine the weights after the last identified weight (-0.1) smaller than the threshold 0.2 among the data with non-zero weights. Since weight (0.86) has already been reflected in operation 401, weights after the finally identified weight (-0.1) can be ignored and not determined, and are not included in the second weight information 415. Furthermore, the electronic device 100 can classify the weight information 405 into data groups 406-3 with weights of 0 and data groups 407-3 with weights of non-zero. Additionally, as... Figure 3 The electronic device 100 can identify the number of zeros in each data group with a weight of 0. The number of zeros corresponding to group 406-3 is nine. Here, the electronic device 100 can calculate the number of zeros by including a weight of a threshold (0.2) or greater in the number of zeros. For example, “0.7” is not zero, but if the weight is 0.2 or greater, the data is determined to be 0; therefore, the number of zeros can be calculated by including “0.7” in the number of zeros.

[0171] Regarding operation 402, electronic device 100 can use the number of zeros obtained as an index. The number of zeros corresponding to group 406-3 is nine, and electronic device 100 can use the number of zeros obtained (nine) as index 411-3 corresponding to group 406-3.

[0172] Regarding operation 402, electronic device 100 can obtain second weight information 415 based on weight information 405, a preset threshold, and the obtained index 411-3. The second weight information 415 may include index 411-3, weight 407-3, and order information (in the order of 411-3 and 407-3).

[0173] Electronic device 100 can classify weight information 405 into multiple classes. Here, a class can refer to a unit used to classify multiple weights for artificial intelligence analysis. Classes can be classified based on the magnitude of the weights (or the importance of the weights). For example, suppose 100 weights are used for artificial intelligence analysis. Weights with a threshold value or greater among the 100 weights may be assigned to the first class, and weights with a threshold value may be assigned to the second class. Meanwhile, the number of classes, the number of thresholds, and the threshold values ​​can vary depending on the embodiment.

[0174] For example, the first category may refer to information related to the first weight information 410 obtained according to the first weight size criterion (e.g., the weight size value is 0.2 or greater), while the second category may refer to information related to the second weight information 415 obtained according to the second weight size criterion (e.g., the weight size value is less than 0.2).

[0175] Electronic device 100 can obtain first and second weight information 410 and 415 corresponding to multiple classes. First weight information 410 can be obtained by grouping data with relatively large weights, and second weight information 415 can be obtained by grouping data with small weights. Because using both first weight information 410 and second weight information 415 is equivalent to using the entire weight information 405, calculation accuracy can be maintained. However, in cases where electronic device 100 has insufficient resources, since both first weight information 410 and second weight information 415 may not be used, calculation operations can be performed using only first weight information 410. When only first weight information 410 with relatively large weight values ​​is used, accuracy may be slightly degraded, but calculation processing operations can still be performed normally.

[0176] Figure 5A It is used to describe based on Figure 4 The diagram illustrates the operation of obtaining the weight matrix by generating information from the zero-jump operation in the embodiment.

[0177] Referring to 5A, the weight information 505 can correspond to Figure 4 The weight information is 405, and the weight information 510 and 515 can correspond to... Figure 4 The first and second weight information 410 and 415 are used. Therefore, the redundant descriptions of weight information 505 and weight information 510 and 515 corresponding to multiple classes are omitted.

[0178] The electronic device 100 can store the obtained weight information 510 and 515 in DRAM 501 and send the weight information 510 and 515 to the kernel generator 520 as needed.

[0179] Kernel generator 520 can be a kernel generation module and can generate new weight information based on at least one received weight information. For example, kernel generator 520 can generate new weight information 525 corresponding to a new class based on first weight information 510 corresponding to a first class and second weight information 515 corresponding to a second class. Here, kernel generator 520 can use the following predefined code information to generate new weight information 525. The predefined code information can be code related to operations that process data using a stack structure. Meanwhile, kernel generator 520 can also be a parameter generator.

[0180] Here, the new weight information 525 generated by the kernel generator 520 can be compared with... Figure 3 The zero-jump operation in both cases obtains the same weight information. The difference lies in... Figure 3 The weight information disclosed in Figure 5 is obtained by performing a zero-jump operation on weight information 306, and the weight information 525 disclosed in Figure 5 is obtained based on weight information 510 and 515 corresponding to multiple classes. Therefore, according to Figure 3 The method cannot obtain new weight information 525 by combining weight information 510 and 515 corresponding to multiple classes. Therefore, the kernel generator 520 can generate integrated weight information by combining individual weight information.

[0181] at the same time, Figure 5A The operation is publicly available. It can be executed using the following code.

[0182] while(not empty){

[0183] for(all classes){

[0184] skip_count=pop(class_id,SKIP);

[0185] if(skip_count <count_min){

[0186] count_min = skip_count;

[0187] class_sel = class_id;

[0188] }

[0189] push(class_id,SKIP,skip_count);

[0190] }

[0191] for(every classes){

[0192] if (class_id != class_sel) {

[0193] skip_count=pop(class_id,SKIP);

[0194] skip_count-=(count_min+1);

[0195] push(class_id,SKIP,skip_count);

[0196] }

[0197] }

[0198] skip_count=pop(class_sel,SKIP);

[0199] kernel_val=pop(class_sel,KERNEL);

[0200] store(skip_count,kernel_val);

[0201] }

[0202] Figure 5B This is a diagram illustrating another example of the operation used to obtain the weight matrix.

[0203] Reference Figure 5BThe electronic device 100 can store weight information 510 and 515 for each class in DRAM 501, and store new weight information 525 generated by the kernel generator 520 in on-chip memory 502. Here, weight information 510 can be first weight information corresponding to the first class, and weight information 515 can be second weight information corresponding to the second class. Furthermore, weight information 525 can be newly generated weight information by combining the first and second classes. Here, according to the embodiment, the user can perform the operation of generating new weight information corresponding to the first class in on-chip memory 502. Here, weight information 525 can include indices 525-1, 525-3, and 525-5 and weights 525-2, 525-4, and 525-6 corresponding to indices 525-1, 525-3, and 525-5.

[0204] According to an embodiment, in order to generate weight information corresponding to the first class, the kernel generator 520 can receive weight information 525 generated by combining the first class and the second class from the on-chip memory 502, and receive second weight information 515 corresponding to the second class.

[0205] Here, weight information 515 may include index 515-1 and weight 515-2 corresponding to index 515-1. As an example, kernel generator 520 may receive only index 515-1 of weight information 515 from DRAM 501. Because information about the weights is already stored in on-chip memory 502, only index 515-1 needs to be received, without needing to receive weight 515-2 separately. Furthermore, kernel generator 520 may generate weight information 530 corresponding to the first type based on the received index 515-1 and the received weight information 525. Weight information 530 may include indices 530-1 and 530-3, and weights 530-2 and 530-4 corresponding to indices 530-1 and 530-3. Additionally, kernel generator 520 may erase the weight corresponding to "9" corresponding to the received index 515-1 from weight information 525. Calculations can be performed by adding the values ​​of the indices themselves in weight information 525 and converting the portion corresponding to the weight to "1". Specifically, kernel generator 520 can check for the quantity 9 by adding the "3" at index 525-1, the transformed value "1" of weight 525-2, and the "5" at index 525-3, and can erase the "-0.1" of weight 525-4 as the later value. Kernel generator 520 can obtain "10" by adding all of the "5" at index 525-3, the transformed value "1" of the erased weight 525-4, and the "4" at index 525-5, and "10" can be used as the new index 530-3.

[0206] According to another embodiment, the kernel generator 520 can erase the weight information 525 stored in the on-chip memory 502 and receive the first weight information corresponding to the first type from the DRAM 501. The first weight information 510 can be the same as the newly generated weight information 530.

[0207] at the same time, Figure 5B The publicly available operations can be used Figure 5A The code described in the document will be used for execution. Redundant code descriptions have been omitted.

[0208] Figure 6A This is a diagram used to illustrate various examples of obtaining the weight matrix.

[0209] Reference Figure 6A The electronic device 100 may include at least one of DRAM 601, direct memory access controller (DMAC) 602, kernel generator 603, or on-chip memory 604. Furthermore, the electronic device 100 may obtain final weight information based on at least one of the aforementioned components and perform artificial intelligence analysis based on the final weight information.

[0210] Electronic device 100 can perform operations to generate or change weight information according to various examples.

[0211] In the first example 605, it is assumed that the weight information corresponding to the first category is stored in DRAM 601, and only the weight information corresponding to the first category is used to perform artificial intelligence analysis. Because the electronic device 100 only uses the weight information corresponding to the first category to perform artificial intelligence analysis, separate operations to generate or change the weight information may not be required. Therefore, the kernel generator 603 does not need to generate new weight information. The electronic device 100 can send the weight information corresponding to the first category stored in DRAM 601 to DMAC 602, and directly send the weight information corresponding to the first category sent to DMAC 602 to on-chip memory 604. Furthermore, the electronic device 100 can perform artificial intelligence analysis by using the weight information corresponding to the first category sent to on-chip memory 604. When a fast data processing speed is required, the data processing method according to the first example 605 can be used.

[0212] In the second example 610, it is assumed that weight information corresponding to the first category and weight information corresponding to the second category are stored in DRAM 601, and both weight information corresponding to the first category and weight information corresponding to the second category are used to perform artificial intelligence analysis. Since only weight information corresponding to the first category and weight information corresponding to the second category are stored in DRAM 601, and there is no integrated weight information obtained by combining the first and second categories, the electronic device 100 can generate integrated weight information. To generate integrated weight information, the electronic device 100 can send the weight information corresponding to the first category and weight information corresponding to the second category stored in DRAM 601 to DMAC 602, and DMAC 602 can send back the weight information corresponding to the first category and weight information corresponding to the second category to kernel generator 602. Kernel generator 603 can generate new weight information (integrated weight information) based on the received weight information corresponding to the first category and the received weight information corresponding to the second category. Furthermore, the electronic device 100 can send integrated weight information generated by the kernel generator 603 to the on-chip memory 604 and perform artificial intelligence analysis based on the integrated weight information sent to the on-chip memory 604. When high-quality data analysis results are required, the data processing method according to the second example 610 can be used.

[0213] In the third example 615, it is assumed that the weight information corresponding to the second category is stored in DRAM 601, and the weight information corresponding to the first category is stored in on-chip memory 604. Artificial intelligence analysis is performed using both the weight information corresponding to the first category and the weight information corresponding to the second category. Here, since the electronic device 100 has not yet obtained the weight information obtained by combining the first and second categories as a result, the electronic device 100 needs to generate integrated weight information. Therefore, the electronic device 100 can send the weight information corresponding to the second category stored in DRAM 601 to DMAC 602, and send the weight information corresponding to the second category that was sent to DMAC 602 to kernel generator 603. Furthermore, the electronic device 100 can send the weight information corresponding to the first category stored in on-chip memory 604 to kernel generator 603. Furthermore, the electronic device 100 can generate integrated weight information through kernel generator 603 based on the weight information corresponding to the first category and the weight information corresponding to the second category. Furthermore, the electronic device 100 can send the generated integrated weight information from kernel generator 603 to on-chip memory 604. Furthermore, the electronic device 100 can perform artificial intelligence analysis based on the integrated weight information sent to the on-chip memory 604. When changing from a fast data processing method to a high-quality data processing method, the data processing method according to the third example 615 can be used.

[0214] In the fourth example 620, it is assumed that the weight information corresponding to the second category is stored in DRAM 601, and the integrated weight information corresponding to both the first and second categories is stored in on-chip memory 604. Artificial intelligence analysis is performed using only the weight information corresponding to the first category. Currently, since on-chip memory 604 only stores the integrated weight information, electronic device 100 needs to generate the weight information corresponding to the first category via kernel generator 603. Therefore, electronic device 100 can send the integrated weight information from on-chip memory 604 to kernel generator 603. Furthermore, electronic device 100 can send the weight information corresponding to the second category from DRAM 601 to DMAC 602, and send the sent weight information corresponding to the second category to kernel generator 603. Electronic device 100 can generate weight information corresponding to the first category based on the integrated weight information and the weight information corresponding to the second category. Because the integrated weight information includes both the weight information corresponding to the first category and the weight information corresponding to the second category, electronic device 100 can perform an operation to erase only the weight information corresponding to the second category from the integrated weight information. Because kernel generator 603 performs a data erasure operation, kernel generator 603 can correspond to an erasure mode. Furthermore, electronic device 100 can perform artificial intelligence analysis using only the weight information corresponding to the first class generated by kernel generator 603. When changing from a high-quality data processing method to a fast data processing method, the data processing method according to fourth example 620 can be used.

[0215] In the fifth example 625, it is assumed that weight information corresponding to the first class and weight information corresponding to the second class are stored in on-chip memory 604, respectively, and artificial intelligence analysis is performed using both the weight information corresponding to the first class and the second class. Electronic device 100 can send the weight information corresponding to the first class and the weight information corresponding to the second class from on-chip memory 604 to kernel generator 603 to generate integrated weight information. Furthermore, electronic device 100 can generate integrated weight information based on the weight information corresponding to the first class and the weight information corresponding to the second class by kernel generator 603. Furthermore, electronic device 100 can send the integrated weight information generated by kernel generator 603 to on-chip memory 604. Furthermore, electronic device 100 can perform artificial intelligence analysis based on the integrated weight information. The data processing method according to the fifth example 625 can be used when changing from a fast data processing method to a high-quality data processing method.

[0216] Figure 6B This is a diagram used to illustrate other examples of obtaining the weight matrix.

[0217] Reference Figure 6B ,like Figure 6AThe electronic device 100 may include at least one of DRAM 601, direct memory access controller (DMAC) 602, kernel generator 603, or on-chip memory 604.

[0218] The sixth example 630 could be an example similar to the fourth example 620, where weight information corresponding to the second category is stored in DRAM 601, weight information corresponding to the combined weights of both the first and second categories is stored in on-chip memory 604, and artificial intelligence analysis is performed using only the weight information corresponding to the first category. However, the kernel generator 603 of the sixth example 630 may not receive all the weight information corresponding to the second category from DRAM 601, but only the index. Because the weights corresponding to the weight information of the second category have already been received from on-chip memory 604, only the index, excluding the weights, can be received from DRAM 601. (See also...) Figure 5B It describes the specific operations associated with it.

[0219] The seventh example 635 could be an example where weight information corresponding to the third category is stored in DRAM 601, and integrated weight information corresponding to all first, second, and third categories is stored in on-chip memory 604, using only the weight information corresponding to the first and second categories to perform artificial intelligence analysis. Currently, since only integrated weight information is stored in on-chip memory 604, electronic device 100 needs to generate weight information corresponding to the first and second categories via kernel generator 603. Therefore, electronic device 100 can execute control to send integrated weight information from on-chip memory 604 to kernel generator 603. Furthermore, electronic device 100 can execute control to send only the index of the weight information corresponding to the third category stored in DRAM 601 to kernel generator 603 via DMAC 602. Additionally, kernel generator 603 can generate new weight information using the received integrated weight information and index. See below for further details. Figure 10B Describe the specific operations related to this.

[0220] The eighth example 640 may be an example where weight information corresponding to the second category and weight information corresponding to the third category are stored in DRAM 601, and integrated weight information corresponding to all first, second, and third categories is stored in on-chip memory 604, and artificial intelligence analysis is performed using only the weight information corresponding to the first category. Currently, since only integrated weight information is stored in on-chip memory 604, electronic device 100 needs to generate weight information corresponding to the first category via kernel generator 603. Therefore, electronic device 100 can execute control to send integrated weight information from on-chip memory 604 to kernel generator 603. Additionally, electronic device 100 can execute control to send only the indexes of the weight information corresponding to the second category and the indexes of the weight information corresponding to the third category stored in DRAM 601 to kernel generator 603 via DMAC 602. Furthermore, kernel generator 603 can generate new weight information using the received integrated weight information and indexes. See below for further details. Figure 10C Describe the specific operations related to this.

[0221] Figure 7 This is a diagram used to describe the general method 705, the pruning operation 710, and the zero-jump operation 715.

[0222] Reference Figure 7 The electronic device 100 can multiply the weight information 706 and the input data 707 according to the general method 705. The general method 705 can be used without applying any modification to the weight information 706 and can be a method that uses the obtained weights as is.

[0223] Simultaneously, the electronic device 100 can consider information about the magnitude of the weights in the weight information 706 to modify the weight information 706. Specifically, the electronic device 100 can perform a pruning operation 710. The pruning operation 710 can be an operation that changes the weights to 0 when the magnitude of the weight is a threshold or smaller. For example, suppose the threshold is 0.09. The electronic device 100 can change the weights with a magnitude of the threshold (0.09) or smaller to 0. The electronic device 100 can group the weights with a magnitude of the threshold (0.09) or smaller into consecutive weights in the weight information 706, and can obtain multiple groups 706-1, 706-2, and 706-3. Furthermore, the electronic device 100 can change the weights in each of the multiple groups 706-1, 706-2, and 706-3 to 0. Such a pruning operation 710 can have the purpose of simplifying the calculation process by changing a certain magnitude or smaller weight to 0. Therefore, the electronic device 100 can obtain the weight information 711 changed by the pruning operation 710. Furthermore, the electronic device 100 can multiply the changed weight information 711 with the input data 707. The general method 705 and the calculation method via the pruning operation 710 have the same number of calculations, but in the calculation method via the pruning operation 710, the calculation process can be simplified and the processing speed can be improved because the number of times multiplied by 0 is larger.

[0224] Simultaneously, the electronic device 100 can perform a zero-jump operation 715 on the changed weight information 711 generated by the pruning operation 710. Because it has already referenced... Figure 3 The zero-jump operation is described, so redundant descriptions are omitted. The electronic device 100 can generate weight information 716 by performing a zero-jump operation 715 on the changed weight information 711, and the weight information 716 may include indices 3, 5 and 4, weights 0.7, -0.1 and 0.86, and sequence information 3, 0.7, 5, -0.1, 4 and 0.86.

[0225] Figure 8 This is a diagram illustrating an example of obtaining weight information.

[0226] Reference Figure 8 The electronic device 100 can obtain multiple classes and weight information corresponding to each of the multiple classes by classifying the weight information 805 according to preset criteria. Because it has already referenced... Figure 4 The operation of obtaining multiple classes and their corresponding weight information is described in detail, so redundant descriptions will be omitted.

[0227] Here, the weight information 805 can correspond to the result data to which a pruning operation has been performed. The weight information 805 can include data with a weight of 0 and data with a weight of non-zero, namely 805-1, 805-2, 805-3, 805-4, and 805-5.

[0228] Here, the electronic device 100 can classify the weight information 805 into three categories. A first threshold and a second threshold can be pre-stored to classify the three categories. When the magnitude (or importance) of the weight is at or above the first threshold, the electronic device 100 determines that the weight corresponds to the first category; when the magnitude (importance) of the weight is at or above the second threshold but less than the first threshold, it determines that the weight corresponds to the second category; and when the magnitude (importance) of the weight is less than the second threshold, it determines that the weight corresponds to the third category.

[0229] For example, suppose the first threshold is 0.6 and the second threshold is 0.3. Because the magnitude of each of weights (0.7)805-1 and (0.8)805-4 is equal to or greater than the first threshold (0.6), the electronic device 100 can determine that weight (0.7)805-1 and weight (0.8)805-4 correspond to the first class. Furthermore, because the magnitude of each of weights (0.5)805-3 and (-0.4)805-5 is equal to or greater than the second threshold (0.3) and less than the first threshold (0.6), the electronic device 100 can determine that weights (0.5)805-3 and weight (-0.4)805-5 correspond to the second class. Additionally, because the magnitude of weight (-0.1)805-2 is less than the second threshold (0.3), the electronic device 100 can determine that weight (-0.1)805-2 corresponds to the third class. The first class can be a set of weights with the highest magnitude, while the third class can be a set of weights with the lowest magnitude.

[0230] The electronic device 100 can obtain weight information corresponding to each class. Specifically, the electronic device 100 can obtain weight information 811 corresponding to the first class, weight information 812 corresponding to the second class, and weight information 813 corresponding to the third class. The weight information 811 corresponding to the first class can be obtained by considering weights other than those determined to be in the first class (805-1 and 805-4) as 0. The weight information 812 corresponding to the second class can be obtained by considering weights other than those determined to be in the second class (805-3 and 805-5) as 0. The weight information 813 corresponding to the third class can be obtained by considering weights other than those determined to be in the third class (805-2) as 0.

[0231] At the same time, the weight information corresponding to each class may not include the number of zeros placed after the last weight of each class. For example, the weight information 811 corresponding to the first class may only include the data up to the last weight 805-4, and the weight information 813 corresponding to the third class may only include the data up to the last weight 805-2.

[0232] At the same time, since it has already been referenced Figure 4 The method for obtaining the weight information corresponding to each class is described, with redundant descriptions omitted.

[0233] Electronic device 100 may include at least one of DRAM 810 or kernel generator 815. Electronic device 100 may store weight information 811 corresponding to a first class, weight information 812 corresponding to a second class, and weight information 813 corresponding to a third class in DRAM 810. Furthermore, electronic device 100 may perform artificial intelligence analysis as needed by using at least one of the weight information 811 corresponding to the first class, weight information 812 corresponding to the second class, or weight information 813 corresponding to the third class.

[0234] exist Figure 8 In this context, it is assumed that only the weight information 811 corresponding to the first category is used to perform artificial intelligence analysis. Electronic device 100 can perform artificial intelligence analysis by using only the weight information 811 corresponding to the first category from among multiple weight information stored in DRAM 810. Here, electronic device 100 can send the weight information 811 corresponding to the first category to kernel generator 815. However, since DRAM 810 itself already includes the weight information 811 corresponding to the first category, it may not be necessary to generate new weight information. Since kernel generator 815 does not need to perform separate combination operations, when high data processing speed is required, it can use... Figure 8 The data processing methods disclosed in the example are as follows.

[0235] At the same time, artificial intelligence analysis is performed using only the weight information 811 corresponding to the first category. Figure 8 Examples in the text can be similar to Figure 6A The first example is 605. However, Figure 8 The examples published in the document can be categorized into three classes, unlike... Figure 6A The first example is 605.

[0236] Figure 9 This is a diagram illustrating an example of generating weight information.

[0237] Reference Figure 9 The weight information 905 can correspond to Figure 8The weight information is 805. Additionally, the weight information 911 corresponding to the first class, the weight information 912 corresponding to the second class, and the weight information 913 corresponding to the third class can respectively correspond to... Figure 8 The weight information is 811, 812, and 813. Therefore, redundant descriptions have been omitted.

[0238] exist Figure 9 In the example, it is assumed that weight information 911 corresponding to the first class, weight information 912 corresponding to the second class, and weight information 913 corresponding to the third class are stored in DRAM 601, and all of the weight information corresponding to the first, second, and third classes are used to perform artificial intelligence analysis. Here, the new weight information obtained by integrating all of the weight information corresponding to the first, second, and third classes can be described as integrated weight information 914.

[0239] Electronic device 100 can send each of the weight information 911, 912, and 913 stored in DRAM 910 to kernel generator 915 to combine the weight information 911 corresponding to the first class, the weight information 912 corresponding to the second class, and the weight information 913 corresponding to the third class. Kernel generator 915 can generate integrated weight information 914 based on each of the weight information 911, 912, and 913. Here, kernel generator 915 can use predefined code information to generate integrated weight information 914. The predefined code information can be code related to operations that process data using a stack structure. When high quality is requested for the data analysis results, it can be used according to... Figure 9 The data processing methods disclosed in the example are as follows.

[0240] At the same time, by using Figure 9 An example of using integrated weighting information in 914 to perform artificial intelligence analysis could be similar to... Figure 6A The second example is 610. However, Figure 9 The examples published in the document can be categorized into three classes, unlike... Figure 6A The second example is 610.

[0241] at the same time, Figure 9 The publicly available operations can be used Figure 5A The code described in the document will be used for execution. Redundant descriptions of the code have been omitted.

[0242] Figure 10A This is a diagram illustrating another example of how weight information is generated.

[0243] Reference Figure 10A The weight information 1005 can correspond to Figure 8The weight information 805. Additionally, the weight information 1011 corresponding to the first class, the weight information 1012 corresponding to the second class, and the weight information 1013 corresponding to the third class can respectively correspond to... Figure 8 The weight information 811, 812, and 813. Furthermore, the integrated weight information 1014 corresponding to all of the first, second, and third categories can correspond to... Figure 9 The integrated weight information 914. Therefore, it will be omitted. Figure 8 and Figure 9 Redundant description.

[0244] exist Figure 10A In the example, it is assumed that weight information 1011 corresponding to the first class, weight information 1012 corresponding to the second class, and weight information 1013 corresponding to the third class are stored in DRAM 1010, and integrated weight information 1014 is stored in on-chip memory 1020. Furthermore, in Figure 10A In the example, it is assumed that artificial intelligence analysis is performed using weight information corresponding to the first and second categories. Here, weight information 1013 may include index 1013-1 and weight 1013-2 corresponding to index 1013-1. Furthermore, weight information 1014 may include indices 1014-1, 1014-3, 1014-5, 1014-7, and 1014-9, and weights 1014-2, 1014-4, 1014-6, 1014-8, and 1014-10 corresponding to indices 1014-1, 1014-3, 1014-5, 1014-7, and 1014-9. The integrated weight information may refer to all weight information from the first, second, and third categories.

[0245] according to Figure 10A For example, electronic device 100 stores only weight information 1011 corresponding to the first class, weight information 1012 corresponding to the second class, weight information 1013 corresponding to the third class, and integrated weight information 1014, but may not store weight information corresponding to the first and second classes. Therefore, electronic device 100 can generate weight information 1016 corresponding to the first and second classes through kernel generator 1015. Here, weight information 1016 may include indices 1016-1, 1016-3, 1016-5, and 1016-7 and weights 1016-2, 1016-4, 1016-6, and 1016-8 corresponding to indices 1016-1, 1016-3, 1016-5, and 1016-7.

[0246] Here, kernel generator 1015 can receive from DRAM 1010 both index 1013-1 and weight 1013-2 corresponding to index 1013-1 in the weight information 1013 of the third type. In addition, kernel generator 1015 can receive from on-chip memory 1020 indexes 1014-1, 1014-3, 1014-5, 1014-7 and 1014-9 and weights 1014-2, 1014-4, 1014-6, 1014-8 and 1014-10 corresponding to indices 1014-1, 1014-3, 1014-5, 1014-7 and 1014-9 in the integrated weight information 1014.

[0247] Here, kernel generator 1015 can generate weight information 1016 corresponding to the first and second classes based on integrated weight information 1014 and weight information 1013 corresponding to the third class. Because integrated weight information 1014 includes all of the weight information 1011 corresponding to the first class, weight information 1012 corresponding to the second class, and weight information 1013 corresponding to the third class, electronic device 100 can perform an operation that erases only the weight information corresponding to the third class from the integrated weight information 1014. Because kernel generator 1015 performs a data erasure operation, kernel generator 1015 can correspond to an erasure mode. Furthermore, electronic device 100 can perform artificial intelligence analysis by using only the weight information 1016 corresponding to the first and second classes generated by kernel generator 1015. When fast data processing speed and high-quality data analysis results are required, it can be used according to... Figure 10A The example data processing method.

[0248] Figure 10B It is used to describe in Figure 10A The example illustrates the operation of applying another calculation procedure.

[0249] Reference Figure 10B Kernel generator 1015 is executing with Figure 10A During the same operation, weight 1013-2 may not be received from DRAM 1010. Specifically, kernel generator 1015 may only receive index 1013-1 from DRAM 1010. As a result, the operation of subtracting weights during the generation of new weight information 1016 can be performed only. Therefore, kernel generator 1015 can find the location of the weight (-0.1) erased only by index 1013-1. Thus, unlike... Figure 10A For example, kernel generator 1015 can receive only the index 1013-1 of weight information 1013 from DRAM 1010.

[0250] because Figure 10B The remaining operations can correspond to Figure 10A Those, therefore, redundant descriptions have been omitted.

[0251] Figure 10C This is a diagram illustrating yet another example of how weight information is generated.

[0252] Reference Figure 10C DRAM 1010 may include weight information 1011 corresponding to the first type, weight information 1012 corresponding to the second type, and weight information 1013 corresponding to the third type. Here, weight information 1012 may include indices 1012-1 and 1012-3, and weights 1012-2 and 1012-4 corresponding to indices 1012-1 and 1012-3. Furthermore, weight information 1013 may include index 1013-1 and weight 1013-2 corresponding to index 1013-1.

[0253] Kernel generator 1015 can receive only indices 1012-1 and 1012-3 of weight information 1012 and index 1013-1 of weight information 1013 from DRAM 1010. In addition, kernel generator 1015 can receive indices 1014-1, 1014-3, 1014-5, 1014-7 and 1014-9 from integrated weight information 1014 from on-chip memory 1020, as well as weights 1014-2, 1014-4, 1014-6, 1014-8 and 1014-10 corresponding to indices 1014-1, 1014-3, 1014-5, 1014-7 and 1014-9.

[0254] Here, kernel generator 1015 can generate new weight information 1017 by erasing the weights corresponding to indices 1012-1 and 1012-3 of weight information 1012 and index 1013-1 of weight information 1013 from the received weight information 1014. Here, weight information 1017 may include indices 1017-1 and 1017-3, and weights 1017-2 and 1017-4 corresponding to indices 1017-1 and 1017-3. Weight information 1017 may refer to weight information corresponding to the first type, and therefore may be the same as weight information 1011.

[0255] because Figure 10C The remaining operations can correspond to Figure 10A and Figure 10B Those, therefore, redundant descriptions have been omitted.

[0256] Figure 11 It is a diagram used to describe the distribution of weights that change based on pruning operations.

[0257] Reference Figure 11Organization chart 1105 can be an organization chart corresponding to basic weight information, and organization chart 1125 can be an organization chart after pruning. Organization charts 1105 and 1125 can represent the relationship between the input and output layers. Furthermore, the pruning operation can be performed using an artificial intelligence model.

[0258] Assume the input layer comprises 4 nodes i1, i2, i3, and i4, and the output layer comprises 5 nodes o1, o2, o3, o4, and o5. Here, weights can be determined between the nodes in the input and output layers, and in the above assumption, a total of 20 weights can be determined. In organization charts 1105 and 1125, thick solid lines can represent data with non-zero weights, thin solid lines can represent data with zero weights, and dashed lines can represent data for which pruning operations have been performed.

[0259] Table 1110 may refer to basic weight information, and Table 1130 may refer to weight information after pruning operations are performed on the basic weight information. The AI ​​model can identify non-zero weights from the 20 weights included in Table 1110, and can also identify weights less than a preset threshold (0.1) from these non-zero weights. The AI ​​model can group the non-zero weights less than the preset threshold (0.1) into a group 1111. Furthermore, the AI ​​model can perform a pruning operation on the identified group 1111. Here, pruning can refer to changing non-zero data to zero. After performing the pruning operation on the identified group 1111, all weights in data group 1131 may be zero.

[0260] Table 1115 may represent the number of each weight size of the basic weight information, and Table 1135 may represent the number of each weight size of the weight information to which the pruning operation is performed. Table 1115 may correspond to Table 1110, and Table 1135 may correspond to Table 1130.

[0261] Graph 1120 can represent the relationship between the magnitude and number of weights in the basic weight information, and graph 1140 can represent the relationship between the magnitude and number of weights in the weight information after the pruning operation. According to graph 1120, the basic weight information can have a distribution where the number of weights decreases as the magnitude of the weights increases, and the number of weights increases as the magnitude of the weights decreases. Furthermore, region 1121 can correspond to group 1111, and region 1141 can correspond to group 1131. Meanwhile, point 1142 can indicate the result of reflecting an increase in the number of zeros based on the pruning operation.

[0262] Figure 12 This is a diagram used to illustrate an example of determining weights without taking into account the resources of electronic devices.

[0263] Reference Figure 12 The total number of weights included in the weighting information can be fixed. If the total number of weights is fixed, then the accuracy of the AI ​​module including the weights may also be constant.

[0264] Table 1205 shows the relationship between the number of weights and the accuracy of the artificial intelligence module. If the total number of weights is fixed at n1, then the accuracy of the artificial intelligence module may also be fixed. Assume that electronic device 100 includes an artificial intelligence module for recognizing faces, and the number of faces in the analysis constraint is 7.

[0265] In the first example 1210, it is assumed that the number of target faces is 4. Since the number of faces in the analysis limit of the artificial intelligence module is 7, the electronic device 100 can analyze all four target faces, which is less than the number of faces in the analysis limit. The first example 1210 can be an example corresponding to a situation where the electronic device 100 has sufficient resources.

[0266] In the second example 1215, assume the number of target faces is 15. Since the number of faces in the analysis limit of electronic device 100 is 7, electronic device 100 may not analyze all target faces. Therefore, electronic device 100 can identify 7 of the 15 target faces and perform analysis only on 7 target faces, while omitting the analysis of the remaining 8 target faces. The second example 1215 may be an example corresponding to a situation where electronic device 100 has insufficient resources.

[0267] Figure 13 Figure 1305 is an example used to describe how weights are determined by considering the resources of an electronic device.

[0268] Reference Figure 13 The total number of weights included in the weighting information can be changed. If the total number of weights is changed, the accuracy of the AI ​​module may change. If the total number of weights decreases, the accuracy of the AI ​​module may decrease, and if the total number of weights increases, the accuracy of the AI ​​module may increase.

[0269] Assume that the total number of current weights of the artificial intelligence modules included in the electronic device 100 is n1, and the number of faces in the analysis constraint is 7.

[0270] In the first example 1310, it is assumed that the number of target faces is 4. Since the number of faces in the analysis limit of the electronic device 100 is 7, the electronic device 100 can analyze all four target faces, which is less than the number of faces in the analysis limit. The first example 1310 can be an example corresponding to a situation where the electronic device 100 has sufficient resources.

[0271] In the second example 1315, assume the number of target faces is 15. Since the number of faces in the analysis limit of the artificial intelligence module is 7, the electronic device 100 may not analyze all target faces. Therefore, the electronic device 100 can change the total number of weights to analyze all 15 target faces. The electronic device 100 can change the total number of weights to n2, which is less than n1. As the total number of weights decreases, the computational cost decreases, and as a result, the resources of the electronic device 100 can be secured. Alternatively, if the total number of weights is n2, assuming the number of faces in the analysis limit is 15, then the electronic device 100 can analyze all target faces (15) by changing the information about the total number of weights of the artificial intelligence module from n1 to n2, where n2 is a value less than n1.

[0272] However, the accuracy of the artificial intelligence module may decrease due to the reduction in the total number of weights. Considering both the user's required accuracy and the resource information of the electronic device 100, the electronic device 100 can identify the optimal operating point among multiple operating points. Based on the identified optimal operating point, the electronic device 100 can change the total number of weights from n1 to n2.

[0273] Figure 14A It is a diagram used to describe the operation of classifying multiple weights into multiple classes.

[0274] Reference Figure 14A The electronic device 100 can pre-train the weight information before performing artificial intelligence analysis. Here, the pre-trained weight information can be generated on an external server other than the electronic device 100. Furthermore, the external server can classify the weight information into multiple classes. The number of classes can be predefined, and the training method can also be predefined.

[0275] Figure 14A The publicly available tables 1405, 1410, 1415-1, 1415-2, 1420, 1425, 1430 and 1435 can represent the relationship between the magnitude of the weights and the number of weights.

[0276] In addition, in order to classify the weight information into multiple classes, the external server may include the first stage 1401 and the second stage 1402.

[0277] In the first stage 1401, the weight information can be classified into class A1 and class B1. According to Table 1405, class A1 can be a class with relatively large weights (or weight importance), and class B1 can be a class with relatively small weights (or weight importance). The criteria for classifying class A1 and class B1 can be preset information or user-input information. For example, the preset information can be a criterion of classifying the first class as the top 20% and classifying the second class as from the top 20% to the top 100%. The external server can group the weights corresponding to the top 20% into class A1 and the weights corresponding to the top 20% to the top 100% into class B1 based on the size (or importance) of all weights. Here, when classifying all weights into class A1 and class B1, according to an embodiment, the values ​​of the existing weights can be changed based on the training results. However, according to another embodiment, classifying the weight information into multiple classes can be implemented by performing only the grouping operation while maintaining the values ​​of the existing weights.

[0278] Furthermore, the external server can perform a pruning operation (S1406) on the class (class B1) with a relatively small weight among multiple classes. If a pruning operation is performed on class B1, the weights with relatively small weights can be changed to 0, and the relationship graph corresponding to the change can be graph 1410. Graph 1410 can indicate that when a pruning operation is performed on the weight information, only the relatively large weights remain.

[0279] Furthermore, the external server can perform a retraining operation (S1411-1) on the remaining class (class A1) in the weight information. The relationship diagram corresponding to the result of the retraining operation S1411-1 can be Figure 1415-1. Here, the retraining operation S1411-1 can refer to the operation of retraining the weights included in class A1. Based on the operation of retraining the weights, the weights included in the existing class A1 can be changed, and the changed weights can be described as class A2. That is, the weights of class A1 and class A2 can be different.

[0280] An external server can perform a retraining operation (S1411-2) on class A2. The relational graph corresponding to the result of the retraining operation S1411-2 can be graph 1415-2. Here, the retraining operation S1411-2 can refer to the operation of reclassifying a class into multiple classes. For example, the external server can classify class A2 into class a11 and class a12 based on the retraining operation. The criteria for classifying class A2 into class a11 and class a12 can be preset information or user input information. For example, suppose the criterion for classifying a class into multiple classes is 50% of the target class. The external server can group the weights corresponding to the top 50% of the weights included in class A2 into class a11, and group the weights corresponding to the top 50% to the top 100% into class a12. Here, class a11 and class a12 can each include 10% of the total weights. Here, the retraining operation S1411-2 can include operations that change the values ​​of the weights along with the grouping operation. That is, based on the results of the retraining operation S1411-2, the weights included in class A2 may be different from the weights included in classes a11 and a12.

[0281] Furthermore, the external server can perform a pruning operation (S1406) on the class (class a12) with a relatively small weight among multiple classes. If a pruning operation is performed on class a12, the weight with a relatively small weight can be changed to 0, and the relationship graph corresponding to the change can be graph 1420. Graph 1420 can indicate that when a pruning operation is performed on the weight information, only the relatively large weight remains.

[0282] Furthermore, the external server can perform a retraining operation (S1421) on the remaining class (class a11) in the weight information. Here, the retraining operation can refer to changing the weights included in the existing class to new weights. The relational graph corresponding to the result of the retraining operation S1421 can be graph 1425. For example, the external server can change class a11 to class a13. Here, the weights included in class a13 and the weights included in class a11 can be different. Here, the retraining operation S1421 can be an operation that moves from the first stage 1401 to the second stage 1402.

[0283] Furthermore, the external server can perform a retraining operation on the weight information (S1426). Here, the retraining operation can refer to performing retraining by adding the class (class a12) that has been erased by the pruning operation to the remaining class (class a13) in the weight information. Performing retraining by adding the class (class a12) that has been erased by the pruning operation can refer to changing the weights included in class a12. The relational graph corresponding to the result of the retraining operation S1426 can be graph 1430. For example, the external server can classify classes a13 and a14 based on the retraining operation. Furthermore, class a14 can be the class generated in the operation S1426 of retraining class a12. Specifically, the external server can add class a12 and perform the retraining operation S1426 on class a13 and class a12. Here, when performing the retraining operation S1426, the weights included in class a13 can be left unchanged. An external server can modify only the weights included in class a12, and can further modify the weights included in class a12 to group the modified weights into a new class a14. In other words, an external server can generate a new class a14 by modifying the weights included in class a12. Class a14 can be a class generated in the operation S1426 of retraining class a12, and the weights included in class a14 can be different from those included in class a12.

[0284] Furthermore, the external server can perform a retraining operation on the weight information (S1431). Here, the retraining operation can refer to performing retraining by adding the classes (class B1) that have been erased by the pruning operation to the remaining classes (classes a13 and a14) in the weight information. Performing retraining by adding the classes (class B1) that have been erased by the pruning operation can refer to changing the weights included in class B1. The relationship graph corresponding to the result of the retraining operation S1431 can be graph 1435. For example, the external server can add class B2 to classes a13 and a14 based on the retraining operation. Here, the weight values ​​corresponding to classes a13 and a14 can be changed without retraining operation S1431. The external server performs the retraining operation based on classes a13, a14, and class B1, and can only change the weights corresponding to class B1. Here, a new class B2 can be generated by changing the weights corresponding to class B1. Class B2 can be a class generated in the operation S1431 of retraining class B1, and the weights included in class B2 can be different from the weights included in class B1.

[0285] The external server can ultimately classify the weight information into three groups (class a13, class a14, and class B2), and store these three groups in memory. Additionally, the external server can separately store the weight information corresponding to each of the three groups (class a13, class a14, and class B2) stored in memory.

[0286] Figure 14B It is a diagram used to describe the computational process of classifying multiple weights into multiple classes.

[0287] Reference Figure 14B The external server may include the original weights. Here, the original weights may be represented by the 4*5 matrix shown in Table 1440. The external server may perform 70% pruning operation 1450, 80% pruning operation 1460, and 90% pruning operation 1470 based on the original weights.

[0288] The 70% pruning operation 1450 can refer to converting 70% of the weights to 0, while leaving 30% of the original weights. The 70% pruning operation 1450 can include classification, pruning, and retraining operations. Here, the classification operation can refer to arranging the weights (original or transformed weights) in a preset manner. In the first classification operation, the external server can arrange the original weights as shown in Table 1451 without any separate data transformation. Additionally, the external server can prune the weights corresponding to 70% of the weights in Table 1451. The weights to be pruned can be determined based on the magnitude of the weights. However, various criteria can be applied additionally, without considering only the magnitude of the weights. If 70% pruning is performed, the external server can obtain the weights corresponding to Table 1452. The external server can perform a retraining operation after obtaining the weights corresponding to Table 1452. The external server can obtain the weights corresponding to Table 1453 as the result of the retraining operation. The external server can perform an 80% pruning operation 1460 based on the weights corresponding to Table 1453.

[0289] The 80% pruning operation 1460 can refer to converting 80% of the weights to 0, while retaining 20% ​​of the weights (including 0) from the weights corresponding to Table 1453. The external server can perform a classification operation on the weights corresponding to Table 1453. When performing the classification operation, the external server may change some weights. For example, the external server can change "0.62" in the weights corresponding to Table 1453 to "0.82". The external server can obtain the weights corresponding to Table 1461 based on the classification operation. Additionally, the external server can obtain the weights corresponding to Table 1462 by performing a pruning operation based on the obtained weights corresponding to Table 1461. Furthermore, the external server can obtain the weights corresponding to Table 1463 by performing a retraining operation based on the obtained weights corresponding to Table 1462. The external server can perform a 90% pruning operation 1470 based on the weights corresponding to Table 1463.

[0290] The 90% pruning operation 1470 can refer to converting 90% of the weights to 0, while leaving 10% of the weights (including 0) from the weights corresponding to Table 1463. The external server can perform a classification operation on the weights corresponding to Table 1463. Here, the classification operation can refer to arranging the weights without individually modifying the data. The external server can obtain the weights corresponding to Table 1471 based on the classification operation. Additionally, the external server can perform a pruning operation based on the obtained weights corresponding to Table 1471 to obtain the weights corresponding to Table 1472. Furthermore, the external server can perform a retraining operation based on the obtained weights corresponding to Table 1472 to obtain the weights corresponding to Table 1473. Here, the external server can identify the weights with non-zero values ​​("1.43" and "0.94") in the weights corresponding to Table 1473 as weights corresponding to the first class. Furthermore, the external server can perform a recovery operation 1480 to identify weights corresponding to the second class.

[0291] Figure 14C This is for further description. Figure 14B The diagram illustrates the operation.

[0292] Reference Figure 14C The external server can perform recovery operation 1480 based on the weights corresponding to Table 1473. Specifically, the external server can obtain the weights corresponding to Table 1481 by recovering the weights ("0.71" and "0.85") that were converted to 0 in the 90% pruning operation 1470. Additionally, the external server can obtain the weights corresponding to Table 1482 by performing a retraining operation based on the weights corresponding to Table 1481. Here, some weights can be changed based on the retraining operation. Here, the external server does not change the weights corresponding to the first class ("1.43" and "0.94"), but can only change the weights added in recovery operation 1480 ("0.71" and "0.85") to new weights ("0.66" and "1.10"). Here, the external server can recognize the newly obtained weights ("0.66" and "1.10") during the recovery process as weights corresponding to the second class.

[0293] Simultaneously, the external server can perform a recovery operation 1490 based on the weights corresponding to Table 1482. Specifically, the external server can obtain the weights corresponding to Table 1491 by recovering the weights ("0.30" and "0.41") that were converted to 0 in the 80% pruning operation 1460. Additionally, the external server can obtain the weights corresponding to Table 1492 by performing a retraining operation based on the weights corresponding to Table 1491. Here, some weights can be changed based on the retraining operation. The external server does not change the weights corresponding to the first class ("1.43" and "0.94") and the weights corresponding to the second class ("0.66" and "1.10"), but can only change the weights added in the recovery operation 1490 ("0.30" and "0.41") to new weights ("0.14" and "0.49"). Here, the external server can recognize the newly obtained weights ("0.14" and "0.49") during the recovery process as weights corresponding to the third class.

[0294] As a result, the external server can obtain weights 1495 categorized into multiple classes. Specifically, the weights corresponding to the first class can be "1.43" and "0.94", the weights corresponding to the second class can be "0.66" and "1.10", and the weights corresponding to the third class can be "0.14" and "0.49". The external server can generate weight information by combining the index of the obtained weights 1495 with the weights corresponding to the index. Furthermore, the external server can send the generated weight information to the electronic device 100.

[0295] Figure 14D It is a diagram used to describe the distribution of weights.

[0296] Reference Figure 14D Graph 1496 can represent the distribution of weights. The horizontal axis of Graph 1496 represents the weights, and the vertical axis represents the number of weights. Regarding... Figure 14A and Figure 14B The weights described in the text have different values. The smaller the absolute value of a weight, the greater the number of weights may be; conversely, the larger the absolute value of a weight, the fewer the number of weights may be.

[0297] Figure 15 It is a diagram used to describe several examples of performing artificial intelligence analysis based on weight information that is classified into multiple classes.

[0298] Figure 15The first example 1501 disclosed herein may be an example of a separate kernel generator 1525 and a neural network processor (NPU) 1535. Specifically, the first example 1501 may include an operation 1510 of classifying raw weights into multiple classes, and an operation of determining which class of weights to use based on the weights of the multiple classes and the resources of the electronic device 100. Here, the operation 1510 of classifying and storing weights into multiple classes may be performed on an external server (not shown), and other operations may be performed on the electronic device 100. The electronic device 100 may receive weights classified into multiple classes from the external server (not shown) and store the received weights in DRAM (memory). Meanwhile, the external server (not shown) may refer to a server that performs the computation process to determine the weights used in the artificial intelligence model.

[0299] Regarding operation 1510, which categorizes and stores the original weights into multiple classes, an external server (not shown) can change from environment 1505 (where the original weights are stored) to environment 1506 (where the original weights are categorized into multiple classes). Environment 1505 (where the original weights are stored) can refer to an environment where multiple weights used in the artificial intelligence module are maintained at a preset number. Environment 1506 (where the original weights are categorized into multiple classes) can refer to an environment where initially stored preset weights are categorized into multiple classes based on their magnitude (or importance). Because it has already been referenced... Figures 14A to 14B The detailed operation of environment 1506, which classifies the original weights into multiple classes, is described, so redundant descriptions are omitted. Meanwhile, environment 1505, which stores the original weights, and environment 1506, which classifies the original weights into multiple classes, can be environments of an external server (not shown). The external server (not shown) can send weight information corresponding to each of environments 1505 and 1506 to electronic device 100, and electronic device 100 can store the received weight information in DRAM (not shown). Specifically, electronic device 100 can classify and store the weights into multiple classes in an offline state (an environment not connected to an external network).

[0300] According to the first example 1501, the electronic device 100 may include at least one of DRAM (not shown), a state analysis module 1515 (state analyzer), an operation control module 1520 (operation manager), a kernel generator 1525, an SRAM 1530, or a neural network processor 1535.

[0301] DRAM (not shown) can receive and store weight information corresponding to multiple classes from an external server (not shown). Furthermore, electronic device 100 can send necessary weight information from DRAM (not shown) to kernel generator 1525 based on kernel generator 1525. Additionally, kernel generator 1525 can generate new weight information based on the necessary weight information. Simultaneously, when kernel generator 1525 does not need to change the weight information, electronic device 100 can directly send weights from DRAM (not shown) to SRAM 1530.

[0302] The state analysis module 1515 and the operation control module 1520 can determine whether to generate new weight information by combining weight information or to use existing weight information as is. Specifically, the state analysis module 1515 can analyze the current state of the resources of the electronic device 100. Furthermore, the operation control module 1520 can determine an operation point based on the resources of the electronic device 100 identified by the state analysis module 1515. Selecting an operation point can be an operation of determining how to use weight information corresponding to multiple classes stored in DRAM (not shown).

[0303] As an example, when the current resources of the electronic device 100 are sufficient, the operation control module 1520 can use all weight information corresponding to all classes to perform artificial intelligence analysis. To synthesize the weight information corresponding to all classes and perform artificial intelligence analysis, the operation control module 1520 can determine to combine (or integrate) the weight information corresponding to the first class, the weight information corresponding to the second class, and the weight information corresponding to the third class. Furthermore, the electronic device 100 can send the weight information corresponding to the first class, the weight information corresponding to the second class, and the weight information corresponding to the third class from DRAM (not shown) to the kernel generator 1525, and can generate integrated weight information corresponding to the first, second, and third classes in the kernel generator 1525. Additionally, the electronic device 100 can send the generated integrated weight information from the kernel generator 1525 to SRAM 1530.

[0304] As another example, when the resources of the electronic device 100 are insufficient, the operation control module 1520 can identify the optimal operation point. The optimal operation point can be determined based on at least one of user satisfaction (analysis accuracy) and processing time. Assume that only the weight information corresponding to the first category is used to determine the optimal operation point for performing artificial intelligence analysis. The operation control module 1520 can send a control signal to the DRAM (not shown) to send the weight information corresponding to the first category stored in the DRAM (not shown) to the SRAM 1530, without generating new weight information.

[0305] SRAM 1530 can receive weight information from DRAM (not shown) or kernel generator 1525 and send the weight information to neural network processor 1535. Neural network processor 1535 can perform artificial intelligence analysis based on the received weight information.

[0306] Meanwhile, the second example 1502 could be an example where the kernel generator 1570 is included in the NPU 1565. In the second example 1502, as in the first example 1501, the operation 1510 of classifying and storing the raw weights into multiple classes on an external server can be performed in the same manner, and therefore redundant descriptions are omitted.

[0307] According to the second example 1502, the electronic device 100 may include DRAM 1550, DMAC 1555, SRAM 1560, NPU 1565, and kernel generator 1570. The electronic device 100 can receive weight information classified into multiple categories from an external server (not shown) and store the weight information in DRAM 1550. Furthermore, the electronic device 100 can analyze its resource status and determine an operation point based on a status analysis module (not shown) and an operation control module (not shown). The electronic device 100 can store information about the determined operation point in NPU 1565. Additionally, NPU 1565 can generate a control signal requesting weight information corresponding to the determined operation point. Furthermore, the electronic device 100 can send the generated control signal to DRAM 1550. Additionally, the electronic device 100 can obtain the weight information corresponding to the generated control signal from DRAM 1550 and send the weight information to NPU 1565 via DMAC 1555 and SRAM 1560. Additionally, the NPU 1565 can send the received weight information to the kernel generator 1570. Furthermore, the kernel generator 1570 can generate new weight information corresponding to the determined operation point based on the received weight information.

[0308] Figure 16 This is a diagram illustrating another example of performing artificial intelligence analysis based on weight information that has been classified into multiple classes.

[0309] Reference Figure 16 The electronic device 100 may include at least one of the following: an operation control module 1605, an operation point determination module 1610, a mode determination module 1615, a kernel generator 1620, or a memory 1630. Here, the memory 1630 may be an on-chip memory or SRAM.

[0310] The operation control module 1605 can analyze the resources of the electronic device 100 to determine whether the current resources of the electronic device 100 are sufficient or insufficient. If the resources of the electronic device 100 are insufficient, the operation control module 1605 can use the operation point determination module 1610 to find the optimal operation point. The operation point determination module 1610 can identify the optimal operation point by comparing the currently available resources in the electronic device 100 with the required resources. Specifically, the operation point determination module 1610 can determine which of several classes to use to perform artificial intelligence analysis. When the operation point determination module 1610 determines which class to use to perform artificial intelligence analysis, the kernel generator 1620 can receive weight information corresponding to the class determined by the operation point determination module 1610. In addition, the kernel generator 1620 can generate new weight information based on the received weight information. Furthermore, the kernel generator 1620 can send the generated weight information to the memory 1630.

[0311] After identifying the optimal operation point, the operation control module 1605 can determine whether to add weight information corresponding to the current new class or erase the weight information corresponding to the specific class from the existing weight information. An example of adding weight information corresponding to the current new class can be similar to... Figure 5A and Figure 9 Examples, and examples of erasing weight information corresponding to a specific class from existing weight information, can be similar to... Figure 10A Examples.

[0312] The pattern determination module 1615 can receive control signals based on the current resource state of the electronic device 100 to determine whether an insertion or erasure mode is currently required. In the case of adding weight information corresponding to a new class, the pattern determination module 1615 can determine the insertion mode. The insertion mode can refer to generating new weight information by inserting new weights into existing indices. In the case of erasing weight information corresponding to a specific class from existing weight information, the pattern determination module 1615 can determine the erasure mode. The erasure mode can refer to changing at least one of the existing weights to 0. The pattern determination module 1615 can send corresponding control signals to the kernel generator 1620 according to the determined mode.

[0313] Here, the kernel generator 1620 can be used as a decoder. The decoder can configure classes based on the differences between previously set operation points and currently set operation points. Furthermore, the decoder can decode and combine the weight information for each class. Additionally, the decoder can send data to the kernel generator 1620 according to the pattern determined by the pattern determination module 1615. In the insertion mode, the kernel generator 1620 can send both the index and the weight. Conversely, in the erase mode, the kernel generator 1620 can send only the index. In the insertion mode, the weight is required because the operation of inserting weights needs to be performed; however, in the erase mode, because the weights at specific positions are erased, it is possible that only the index corresponding to the specific position is sent.

[0314] Furthermore, kernel generator 1620 can generate new weight information by performing predetermined operations based on insertion or erasure modes. Additionally, kernel generator 1620 can send the generated weight information to memory 1630. Electronic device 100 can perform artificial intelligence analysis based on the weight information sent to memory 1630.

[0315] Figure 17 This is a diagram used to describe an example of identifying classes by considering resource information.

[0316] Reference Figure 17 The electronic device 100 can identify the class used for artificial intelligence analysis based on internal memory resources 1710 and class-specific resources 1720.

[0317] Assume that the total internal memory resources included in electronic device 100 are 100. Internal memory resources 1710 may include used resources 1711 and available resources 1712. Used resources 1711 may refer to resources currently used by the internal memory, while available resources 1712 may refer to the capacity that can be further processed in the current internal memory.

[0318] Regarding the current internal memory, assume that the used resource 1711 is 40 and the available resource 1712 is 60. Additionally, class-specific resource 1720 (or class-specific resource combination) can refer to resource information based on multiple class combinations. For example, electronic device 100 can store resources 1721 required for performing artificial intelligence analysis using only the first class and resources 1722 required for performing artificial intelligence analysis using both the first and second classes. Here, assume that resource 1721 required for performing artificial intelligence analysis using only the first class is 40, while resource 1722 required for performing artificial intelligence analysis using both the first and second classes is 80.

[0319] Because the available resource 1712 is 60, the electronic device 100 can determine not to process operations that require resources higher than the available resource 1712. Furthermore, the electronic device 100 can process operations that require resources lower than the available resource 1712. The class combination corresponding to resources lower than the available resource 1712 can be determined as the case where only the first class is used. Therefore, the electronic device 100 can perform artificial intelligence analysis using only the first class. In this case, the processing resource 1730 to be processed separately in internal memory can be 40.

[0320] Electronic device 100 can use only the first type to perform artificial intelligence analysis, and while performing artificial intelligence analysis, internal memory resources 1740 can have 80% of the resources 1741 in use and 20% of the resources 1742 available.

[0321] Figure 18 This is a diagram used to describe another example of identifying classes by considering resource information.

[0322] Reference Figure 18 Because internal memory resources 1810, class-specific resources 1820, processing resources 1830, used resources 1811, 1861-1 and 1861-2, and available resources 1812, 1862-1 and 1862-2 can correspond to Figure 17 The internal memory resources 1710, class-specific resources 1720, processing resources 1730, used resources 1711 and 1741, and available resources 1712 and 1742 are omitted, thus redundant descriptions are omitted.

[0323] Assume that the total internal memory resources included in electronic device 100 are 100. Furthermore, regarding the current internal memory, assume that the used resources 1811 are 10, and the available resources 1812 are 90. Additionally, regarding class-specific resources 1820, assume that the resources 1821 required to perform AI analysis using only the first class are 40, and the resources 1822 required to perform AI analysis using both the first and second classes are 80.

[0324] Electronic device 100 can identify class combinations that are less than the available resources 1812 of the current internal memory. Class combinations less than the available resources 1812 can be combinations using the first class or combinations using both the first and second classes. Since the available resources 1812 of the internal memory are sufficient, there may be situations where multiple combinations related to the class combination may all be executed. That is, processing resources 1830 can be 40 in the case of AI analysis using only the first class, and can be 80 in the case of AI analysis using both the first and second classes.

[0325] Therefore, the electronic device 100 can identify a specific combination among multiple combinations based on preset criteria. The preset criteria may be related to user satisfaction, which may be at least one of processing speed or the quality (accuracy) of the processing result.

[0326] As an example, suppose electronic device 100 performs (1840) artificial intelligence analysis where processing speed is important. Electronic device 100 can identify class combinations with low resources (using only the first class) among multiple class combinations. Furthermore, when performing artificial intelligence analysis using only the first class, internal memory resources 1860-1 may have 50% of the resources 1861-1 in use and 50% of the resources 1862-1 available.

[0327] As another example, suppose electronic device 100 performs artificial intelligence analysis (1850) where the quality of the processing result is important. Electronic device 100 can identify class combinations with high resources among multiple class combinations (using both the first and second classes). Furthermore, when performing artificial intelligence analysis using the first and second classes, the internal memory resources 1860-2 may have 90% of the resources 1861-2 in use and 10% of the resources 1862-2 available.

[0328] Figure 19 This is a diagram used to describe yet another example of how resource information is considered when identifying classes.

[0329] Reference Figure 19 Because internal memory resources 1910, class-specific resources 1920, processing resources 1940, used resources 1911 and 1951, and available resources 1912 and 1952 can correspond to Figure 17 The internal memory resources 1710, class-specific resources 1720, processing resources 1730, used resources 1711 and 1741, and available resources 1712 and 1742 are omitted, thus redundant descriptions are omitted.

[0330] Assume the total memory resources included in electronic device 100 are 100. Furthermore, regarding the current internal memory, assume the used resource 1911 is 40, and the available resource 1912 is 60. Additionally, regarding class-specific resources 1920, assume the resource 1921 required to perform AI analysis using only the first class is 40, and the resource 1922 required to perform AI analysis using both the first and second classes is 80.

[0331] When electronic device 100 performs artificial intelligence analysis, in addition to the resources required according to the class composition, there may also be resources 1930 required by specific operations during the artificial intelligence analysis. For example, various functions performed during the artificial intelligence analysis may require additional resources. For example, suppose that during the operation performed in the artificial intelligence analysis, the resources for the first function are 20.

[0332] exist Figure 17 and Figure 18 In this context, only the resources required for a class-specific combination are considered, but the electronic device 100 may also consider the resources required for a specific operation 1930. As a result, performing the first function using only the first class may require a total of 60 resources, and performing the first function using both the first and second classes may require a total of 100 resources.

[0333] Here, electronic device 100 can determine that the first function cannot be performed using either the first or second type of resource because the available resource 1912 is 90, and only the first type of resource is used to perform the first function. Here, the processing resource 1940 associated with operations performed separately in internal memory can be 60.

[0334] Furthermore, when performing the first function during artificial intelligence analysis using only the first type, the internal memory resource 1950 may have 70% of the resources 1951 in use and 30% of the resources 1952 available.

[0335] Figure 20 It is a flowchart used to describe the operation of setting operation points.

[0336] Reference Figure 20 The electronic device 100 can check its resource status (S2005). The electronic device 100 can obtain real-time resource information of the current electronic device 100.

[0337] Furthermore, the electronic device 100 can determine whether the operation has been terminated (S2010). Here, the operation situation can refer to the situation in which the electronic device 100 is currently using resources to perform a specific operation. For example, the operation situation could be the situation in which the electronic device 100 is performing mathematical calculations for artificial intelligence analysis.

[0338] If the operation of electronic device 100 does not terminate, electronic device 100 can determine whether it needs to change the operating point (S2015). However, if the operation of electronic device 100 terminates, electronic device 100 may not be able to determine whether it needs to change the operating point. Here, the operating point may refer to a data processing method or data processing reference value used by electronic device 100 to perform a specific operation using resources.

[0339] For example, a change in the operation point can refer to a change in the data processing method or the data processing reference value. The data processing method can be a method for processing data quickly or a method for processing data to obtain high-quality (accurate) analytical results. Generally, fast or high-quality data processing methods may require significant resources, while slow or low-quality data processing methods may require fewer resources.

[0340] Electronic device 100 can compare the resources of electronic device 100 obtained in step S2005 with the resource requirement of the specific operation to be performed. If the resource requirement of the specific operation to be performed is less than the resources of electronic device 100, then electronic device 100 can perform the specific operation and continue to check the resource status of electronic device 100 (S2016).

[0341] If the resource requirement of a specific operation to be performed exceeds the resources of electronic device 100, electronic device 100 can determine that the operation point needs to be changed. Furthermore, electronic device 100 can generate a cost function. The cost function can refer to a function used to change the operation point. Electronic device 100 can use the cost function to identify the optimal operation point (S2025). Electronic device 100 can identify the operation point that maximizes user satisfaction, and user satisfaction can be at least one of processing speed or the quality of processing results. Each user may value processing speed or high quality. Therefore, user satisfaction may vary for each user and can be changed through settings. Therefore, electronic device 100 can identify the optimal operation point based on predefined user satisfaction. Furthermore, electronic device 100 can perform artificial intelligence analysis by setting the identified optimal operation point (S2030).

[0342] As an example, suppose the artificial intelligence analysis performed by electronic device 100 is a facial recognition operation. Electronic device 100 can detect human faces included in the input data and obtain detailed information about the detected faces (information indicating who the detected faces belong to). Assume that, based on the resources of the artificial intelligence module executed in electronic device 100, only 7 faces can currently be analyzed. However, suppose 15 faces are detected in the input data. Electronic device 100 only analyzes the faces of 7 people and needs to perform analysis on the faces of 8 people after the faces of 7 people have been analyzed. Here, assume that the user considers processing speed more important than the quality of the processing results. Electronic device 100 can identify the optimal operation point and change the existing operation point, thereby enabling the simultaneous analysis of as many faces as possible. Specifically, to improve processing speed, electronic device 100 can perform artificial intelligence analysis using only the weight information corresponding to the smallest class. If only the weight information corresponding to the smallest class is used to perform artificial intelligence analysis, accuracy may decrease, but processing speed may increase. As a result, electronic device 100 can operate according to the user's intent.

[0343] As another example, suppose the artificial intelligence analysis performed by electronic device 100 involves operating in a low-power mode related to power consumption. Electronic device 100 can identify the optimal operating point based on power consumption and the quality of the processing results. Simultaneously, electronic device 100 can identify the optimal operating point based on at least one of the power usage changes according to a decrease in operating frequency and power cutoff.

[0344] As another example, the operation point can be determined based on at least one of accuracy, latency, and duration. The real-time concept may be important for tasks related to autonomous driving or video playback. Therefore, when the task is related to autonomous driving or video playback, the electronic device 100 can identify the operation point by maximizing latency and subsequently accuracy. Here, lower latency indicates higher user satisfaction, and higher accuracy also indicates higher user satisfaction. Furthermore, the electronic device 100 can identify the operation point by setting a minimum latency or minimum accuracy. For example, the electronic device 100 can identify the operation point by considering the fact that the latency is at least 33ms or less.

[0345] Meanwhile, for tasks related to facial recognition, iris recognition, and fingerprint recognition, accuracy may be important. Therefore, when the task involves facial recognition, iris recognition, and fingerprint recognition, the electronic device 100 can identify the operation point by maximizing accuracy and then considering the duration. Here, higher accuracy generally indicates higher user satisfaction. The electronic device 100 can identify the operation point by setting a minimum accuracy. For example, the electronic device 100 can consider a minimum accuracy of 95% or higher when identifying the operation point.

[0346] Figure 21 This is a diagram illustrating a control method for an electronic device according to embodiments of the present disclosure.

[0347] Meanwhile, in the control method of the electronic device 100, first information including weights associated with the first weight matrix and a first index corresponding to the weights can be stored, as well as second information including additional weights for obtaining the second weight matrix and a second index corresponding to the additional weights (S2105). Furthermore, in the control method, based on the resource information of the electronic device 100, the first weight matrix is ​​obtained by loading the first information, or the second weight matrix is ​​obtained by loading the first information and the second information (S2110).

[0348] Here, the control method may also include obtaining output data by inputting input data into an artificial intelligence model using a first weight matrix or a second weight matrix, and obtaining a second weight matrix based on the first weight matrix or obtaining a first weight matrix based on the second weight matrix, depending on changes to the resource information of the electronic device 100.

[0349] Meanwhile, the second weight matrix can be obtained based on the weights and first index included in the first information, the additional weights, and the second index.

[0350] At the same time, the electronic device 100 can obtain second weight information (or second information) corresponding to the second class, and the second weight information may include the weights included in the second class and the second index corresponding to the weights included in the second class.

[0351] According to an embodiment, the electronic device 100 can obtain a first weight matrix based on first weight information. Here, the first weight matrix is ​​obtained using only the first type and can be a matrix reflecting relatively high weights.

[0352] According to another embodiment, the electronic device 100 can obtain a second weight matrix based on first weight information and second weight information. Here, a first type and a second type can be used to obtain the second weight matrix.

[0353] The second weight matrix can be obtained based on the weights, first index, additional weights, and second index included in the first weight information.

[0354] Here, the weights included in the first weight information can refer to the weights included in the first class, and the first index can refer to the index corresponding to the weights included in the first class. Additionally, the additional weights can refer to the weights included in the second class, and the second index can refer to the index corresponding to the weights included in the second class.

[0355] Here, when obtaining the second weight matrix, the second weight matrix can be obtained by combining the additional weights with the weights included in the first weight information based on the first index and the second index.

[0356] The first index and weight included in the first weight information can be information related to the first category, and the second index and additional weight can be related to the second category. As a result, the electronic device 100 can generate new weight information by combining the first weight information and the second weight information, and generate a new weight matrix based on the newly generated weight information.

[0357] Here, when obtaining the second weight matrix, the position of the weight corresponding to the first index in the second weight matrix can be determined based on the first index, and the position of the additional weight corresponding to the second index in the second weight matrix can be determined based on the second index. At least one of the first index or the second index can be changed based on the position of the weight corresponding to the first index and the position of the weight corresponding to the second index, and the second weight matrix can be obtained based on the position of the weight, the position of the additional weight, and the changed index.

[0358] Resource information can refer to information related to the resources of electronic device 100 and the resources required to perform a specific module (or specific operation).

[0359] Here, resource information may include at least one of the following: the internal memory capacity of electronic device 100 (total capacity, used capacity, available capacity), the power state of electronic device 100, the resources of an application running in electronic device 100, or the resources of an artificial intelligence model running in electronic device 100.

[0360] As an example, resource information may include the required resources corresponding to the power state of electronic device 100. When the power state of electronic device 100 is in high-performance mode, the required resources may be large. However, when the power state of electronic device 100 is in power-saving mode, the required resources may be small. Because the required resources may vary depending on the power state of electronic device 100, electronic device 100 can determine whether to load the first information or both the first and second information based on the resource information corresponding to the power state of electronic device 100.

[0361] As another example, resource information may include the resources required by the application running in electronic device 100. When the application running in electronic device 100 analyzes an artificial intelligence module, the required resources may be large. However, when the application running in electronic device 100 is a program related to alarm functions, the required resources may be small. Because the required resources may vary depending on the resources of the application running in electronic device 100, electronic device 100 can determine whether to load first information or both first and second information based on the resource information corresponding to the application running in electronic device 100.

[0362] As another example, the required resources corresponding to the resources of the artificial intelligence model executed in electronic device 100 can vary in the resource information. This variation in the resources of the artificial intelligence model can refer to a change in the analysis operation. In the case of an artificial intelligence model related to image recognition, the required resources may be large. However, in the case of an artificial intelligence model related to text recognition, the required resources may be small. Because the required resources may vary depending on the resources of the artificial intelligence model executed in electronic device 100, electronic device 100 can determine whether to load the first information or both the first and second information based on the resource information corresponding to the resources of the artificial intelligence model executed in electronic device 100.

[0363] At the same time, the electronic device 100 can determine, based on the resource information, whether to use the weight information corresponding to the first category (first information) or the weight information corresponding to the first and second categories (second information).

[0364] Electronic device 100 may have stored information about the necessary resources when using weight information corresponding to the first class and information about the necessary resources when using weight information corresponding to both the first and second classes. Therefore, electronic device 100 can determine whether to use weight information corresponding to the first class or weight information corresponding to both the first and second classes based on the resources required to perform a specific operation.

[0365] As an example, suppose there are two classes. Suppose the resources are 40 when only the first class is used, and 80 when both the first and second classes are used. Additionally, suppose the available resources in the current internal memory are 60. The electronic device 100 can determine which specific class combination (when only the first class is used) has less resources than the available resources (60) in the current internal memory. Available resources can refer to the capacity that can be further processed in the current internal memory, and class-specific resources (or class combination-specific resources) can refer to resource information based on multiple class combinations.

[0366] Various examples related to resource information are in Figures 17 to 19 As described in the text.

[0367] Simultaneously, the electronic device 100 can also implement embodiments that use only the weight information corresponding to the second category. However, embodiments that perform artificial intelligence analysis using only low weights may be uncommon because the first category is a weighted reorganization including relatively high weights, while the second category is a weighted reorganization including relatively low weights. This is because if artificial intelligence analysis needs to be performed with insufficient resources, then the accuracy of the analysis can only be improved by using the class including high weights. Therefore, this disclosure describes in detail an embodiment that uses only the weight information corresponding to the first category, wherein the first category is a weighted reorganization with relatively high weights.

[0368] Examples of changing resource information can include increasing and decreasing resource information. In both cases, the actual problem is likely to occur when resources are increased. Because decreasing resource information increases the available resources of existing internal memory, processing operations are not affected. However, because increasing resource information decreases the available resources, processing capacity issues may arise when performing resource operations exceeding the available resources.

[0369] Therefore, electronic device 100 can determine the available resources of its internal memory and the resources required for artificial intelligence analysis in real time. When the resources required for artificial intelligence analysis exceed the available resources of internal memory, electronic device 100 can obtain a new weight matrix. To obtain the new weight matrix, electronic device 100 can consider class-specific resources and available resources. That is, electronic device 100 can obtain a weight matrix based on the identified class combinations by identifying combinations of classes that can perform artificial intelligence analysis with available resources.

[0370] Meanwhile, examples of increased resource information can include situations where the goals of AI analysis suddenly increase and resources increase, or where a user wants to switch from an AI module with high processing speed to a high-quality AI module. Conversely, examples of decreased resource information can include situations where the goals of AI analysis decrease and resources decrease, or where a user wants to switch from a high-quality AI module to an AI module with high processing speed. (This is already...) Figure 6A The document describes a detailed example of how resources were modified.

[0371] Meanwhile, the first information may include a second threshold or greater weight in the information obtained by pruning weights less than a first threshold, and an index corresponding to the second threshold or greater weight; and the second information may include weights less than a second threshold in the information obtained by pruning, and an index corresponding to the weights less than a second threshold.

[0372] Here, the first information may include weights obtained by performing a first training on weights of a second threshold or greater in the information obtained through pruning, and an index corresponding to the weights obtained by performing the first training. The second information may include weights obtained by performing a second training on weights of a second threshold or greater and weights of less than the second threshold in the information obtained through pruning, and an index corresponding to the weights obtained by performing the second training. The weights of the second threshold or greater may be fixed during the second training. Furthermore, the control method may also include: changing the first information by performing the first training based on the first information; changing the second information by performing the second training based on the first information and the second information; and obtaining a first weight matrix by loading the changed first information or obtaining a second weight matrix by loading the changed first information and the changed second information. Additionally, the control method may further include obtaining weights by performing a first training on weights of a second threshold or greater in the information obtained through pruning; obtaining an index corresponding to the weights obtained through the first training; changing the weights and indices included in the first information to the weights obtained through the first training and the indices corresponding to those weights; obtaining weights less than the second threshold by performing a second training on the weights obtained through the first training and the weights less than the second threshold in the information obtained through pruning; obtaining an index corresponding to the weights less than the second threshold in the weights obtained through the second training; changing the weights and indices included in the second information to the weights less than the second threshold in the weights obtained through the second training and the indices corresponding to those weights less than the second threshold in the weights obtained through the second training; and obtaining a first weight matrix by loading the changed first information or a second weight matrix by loading the changed first information and the changed second information. Furthermore, during the second training, the weights of the second threshold or greater are fixed.

[0373] Meanwhile, the control method may also include obtaining a first weight matrix by erasing additional weights corresponding to the second index from the third information based on the weights included in the third information corresponding to the second weight matrix, the third index corresponding to the weights, and the second index.

[0374] Here, in obtaining the first weight matrix, the position of the additional weight in the second weight matrix can be determined based on the second and third indices. The additional weight can be erased from the third information based on the position of the additional weight. Furthermore, the first weight matrix can be obtained by modifying the index of the weight that is adjacent to the position of the erased additional weight in the third information.

[0375] At the same time, such as Figure 21 The control method of the electronic device shown can be used in having Figure 1 and Figure 2It can be executed on electronic devices with certain configurations, and can also be executed on electronic devices with other configurations.

[0376] Figure 22 This is a diagram illustrating a comparison of delay changes based on a change in the operating point, according to an embodiment of the present disclosure.

[0377] Reference Figure 22 The electronic device 100 can perform convolution operations according to various examples. Example (a) is described in Table 2205, example (b) is described in Table 2210, example (c) is described in Table 2215, and example (d) is described in Table 2220.

[0378] Electronic device 100 can perform operation 2207 of sending parameters corresponding to the convolutional layer from DMA to on-chip memory (SRAM). Furthermore, electronic device 100 can perform convolution operations using the parameters corresponding to the convolutional layer via a kernel generator included in the NPU (2206). The NPU can include a kernel generator and on-chip memory (SRAM). The on-chip memory (SRAM) can store parameter information corresponding to a specific layer, and the kernel generator can use the parameter information stored in the on-chip memory (SRAM) to perform convolution operations.

[0379] Meanwhile, the operations indicated by regions 2206 and 2207 also apply to table 2205 and the remaining tables 2210, 2215 and 2220, and therefore redundant descriptions are omitted.

[0380] at the same time, Figure 22 Tables 2205 to 2220 can be categorized into regular time intervals (parts T01, T12, T23, T34, T45, and T56). For example, part T01 can refer to the time interval from 0 to time point t1. Referring to Table 2205, electronic device 100 can send parameter information (param0) corresponding to the base layer from DMA to the on-chip memory (SRAM) included in the NPU in part T01.

[0381] Furthermore, the electronic device 100 can perform convolution operations in part T12 using the kernel generator included in the NPU, using the parameter information (param0) corresponding to the base layer.

[0382] Furthermore, in part T23, the electronic device 100 can send parameter information (param1) corresponding to the first layer from the DMA to the on-chip memory (SRAM) included in the NPU.

[0383] Furthermore, the electronic device 100 can perform convolution operations in part of T34 using the kernel generator included in the NPU with the parameter information (param1) corresponding to the first layer.

[0384] Because the operations for some T45 and some T56 are repeated from the previous operations, redundant descriptions have been omitted.

[0385] The total operation time corresponding to Example (a) according to Table 2205 can be t6. In Example (a) described in Table 2205, the operation of receiving parameter information from the DMA and the operation of performing convolution operations in the kernel generator included in the NPU are performed in separate time segments, so the processing time can be long.

[0386] Simultaneously, in Example (b) according to Table 2210, the electronic device 100 can perform convolution operations and transmit parameter information operations at the same time. Specifically, the electronic device 100 can perform convolution operations in part T01 using parameter information (param0) corresponding to the base layer through a kernel generator included in the NPU. Furthermore, the electronic device 100 can transmit parameter information (param1) corresponding to the first layer from DMA to on-chip memory (SRAM) included in the NPU in part T01.

[0387] Furthermore, in part T12, the electronic device 100 can perform convolution operations using parameter information (param1) corresponding to the first layer via a kernel generator included in the NPU. Additionally, in part T12, the electronic device 100 can send parameter information (param2) corresponding to the second layer from DMA to on-chip memory (SRAM) included in the NPU.

[0388] Furthermore, the electronic device 100 can perform convolution operations in part T23 using parameter information (param2) corresponding to the second layer via a kernel generator included in the NPU.

[0389] The total operation time corresponding to Example (b) according to Table 2210 can be t3. This can be an operation time smaller than the time in Example (a) of Table 2205.

[0390] On the other hand, example (c) according to Table 2215 is an example for describing the case of changing the operation point. Electronic device 100 can perform convolution operations in part T01 using parameter information (param0) corresponding to the base layer via a kernel generator included in the NPU. Furthermore, electronic device 100 can send parameter information (param1) corresponding to the first layer from DMA to on-chip memory (SRAM) included in the NPU in part T01. Here, it is assumed that there is a first operation point change 2215-1 at time point t1.

[0391] If a first operation point change 2215-1 is detected, the electronic device 100 can send parameter information (param1') corresponding to the changed first layer from DMA to the on-chip memory (SRAM) included in the NPU in part T12.

[0392] Furthermore, in part T23, electronic device 100 can perform convolution operations using parameter information (param1') corresponding to the changed first layer via a kernel generator included in the NPU. Additionally, in part T23, electronic device 100 can send parameter information (param2') corresponding to the changed second layer from DMA to on-chip memory (SRAM) included in the NPU. Here, it is assumed that a second operation point change 2215-2 exists at time point t3.

[0393] If a second operation point change 2215-2 is detected, the electronic device 100 can send parameter information (param2”) corresponding to the changed second layer from DMA to the on-chip memory (SRAM) included in the NPU in part T34.

[0394] Furthermore, the electronic device 100 can perform convolution operations in part T45 using parameter information (param2”) corresponding to the modified second layer via a kernel generator included in the NPU.

[0395] The total operation time corresponding to Example (c) according to Table 2215 can be t5. In Example (c) according to Table 2215, new parameter information can be received from the DMA whenever the operation point changes. In this case, the process of receiving new parameter information may take time.

[0396] On the other hand, Example (d) according to Table 2220 is an example of a process for describing the execution of a convolution operation without receiving new parameter information from the DMA when there is an operation point change.

[0397] Electronic device 100 can perform convolution operations using parameter information (param0) corresponding to the base layer via a kernel generator in the NPU included in section T01. Furthermore, electronic device 100 can send parameter information (param1) corresponding to the first layer from DMA to on-chip memory (SRAM) included in the NPU in section T01. Here, it is assumed that there is a first operation point change 2220-1 at time point t1.

[0398] If a first operation point change 2220-1 is detected, the electronic device 100 can obtain the changed parameter information (param1') of the first layer in part T12 based on the parameter information (param1) corresponding to the first layer through a kernel generator included in the NPU. Furthermore, the electronic device 100 can perform a convolution operation in part T12 using the changed parameter information (param1') of the first layer through a kernel generator included in the NPU. Additionally, the electronic device 100 can send the changed parameter information (param2') of the second layer from DMA to the on-chip memory (SRAM) included in the NPU in part T12. Here, it is assumed that a second operation point change 2220-2 exists at time point t2.

[0399] If a second operation point change 2220-2 is detected, the electronic device 100 can, in part T23, obtain the parameter information (param2") corresponding to the further changed second layer based on the parameter information (param2') corresponding to the changed second layer via a kernel generator included in the NPU. Furthermore, the electronic device 100 can, in part T23, use the parameter information (param2') corresponding to the further changed second layer via a kernel generator included in the NPU. 2 Use ") to perform convolution operations.

[0400] The total operation time corresponding to Example (d) according to Table 2220 can be t3. Unlike Example (c) according to Table 2215, in Example (d), new parameter information may not be received from the DMA. In Example (d), latency can be reduced by generating new parameter information using previously received parameter information.

[0401] On the other hand, the methods according to the various embodiments disclosed above can be implemented in the form of applications that can be installed on existing electronic devices.

[0402] Furthermore, the methods according to the various embodiments disclosed above can be implemented simply by upgrading the software or hardware of existing electronic devices.

[0403] Furthermore, the various embodiments disclosed above can also be performed by an embedded server included in an electronic device, or by an external server of at least one of the electronic device or display device.

[0404] Furthermore, according to embodiments of this disclosure, the various embodiments described above can be implemented by software including instructions stored in a machine-readable storage medium. A machine is a device operable from and according to the stored instructions, and may include an electronic device (e.g., electronic device A) according to the disclosed embodiments. When a processor executes the instructions, the processor may perform the function corresponding to the instructions directly or under the processor's control using other components. Instructions may include code generated or executed by a compiler or interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, the term "non-transitory" means that the storage medium does not include signals and is tangible, but does not distinguish whether data is stored semi-permanently or temporarily in the storage medium.

[0405] Furthermore, according to embodiments of this disclosure, the methods described in the various embodiments above can be included and provided in a computer program product. The computer program product can be traded as a product between a seller and a buyer. The computer program product can be distributed in the form of a device-readable storage medium (e.g., an optical disc read-only memory (CD-ROM)) or distributed online through an app store (e.g., PlayStore™). In the case of online distribution, at least a portion of the computer program product can be stored, at least temporarily, in a storage medium such as the memory of a manufacturer's server, an app store's server, or a relay server, or can be temporarily generated.

[0406] Furthermore, each component (e.g., module or program) according to the various embodiments described above may include a single entity or multiple entities, and some of the sub-components described above may be omitted, or other sub-components may be included in the various embodiments. Alternatively or additionally, some components (e.g., module or program) may be integrated into one entity to perform the same or similar functions performed by the respective components prior to integration. Operations performed by modules, programs, or other components according to the various embodiments may be performed sequentially, in parallel, iteratively, or heuristically, or at least some operations may be performed in a different order or omitted, or additional operations may be added.

[0407] Although embodiments of this disclosure have been described and illustrated above, this disclosure is not limited to the specific embodiments described above, but can be modified by those skilled in the art without departing from the spirit of this disclosure as disclosed in the appended claims. Such modifications should not be interpreted solely from the technical spirit or prospects of this disclosure.

[0408] Although this disclosure has been described using various embodiments, various changes and modifications may be suggested to those skilled in the art. This disclosure is intended to cover such changes and modifications that fall within the scope of the appended claims.

Claims

1. An electronic device, comprising: The memory is configured to store first information and second information; as well as The processor is configured as follows: Obtain resource information of the electronic device, the resource information indicating the currently available resources of the electronic device. Based on the resource information of the electronic device, a first weight matrix is ​​obtained by loading first information, or a second weight matrix is ​​obtained by loading first information and second information. The first information includes weights associated with the first weight matrix and a first index corresponding to the weights. The second information includes additional weights for obtaining the second weight matrix and a second index corresponding to the additional weights. The processor is configured to obtain a second weight matrix by combining additional weights with weights included in the first information based on a first index and a second index.

2. The electronic device as claimed in claim 1, wherein, The processor is configured to: Output data is obtained by feeding input data into an artificial intelligence model that uses either a first weight matrix or a second weight matrix, and Based on the alteration of the resource information of the electronic device, a second weight matrix is ​​obtained based on the first weight matrix, or a first weight matrix is ​​obtained based on the second weight matrix.

3. The electronic device as claimed in claim 1, wherein, The processor is configured to: The position of the weight corresponding to the first index in the second weight matrix is ​​determined based on the first index. The position of the additional weight corresponding to the second index in the second weight matrix is ​​determined based on the second index. Based on the position of the weight corresponding to the first index and the position of the weight corresponding to the second index, change at least one of the first index or the second index, and The second weight matrix is ​​obtained based on the position of the weights, the position of the additional weights, and the changed index.

4. The electronic device as claimed in claim 1, wherein, The resource information includes at least one of the following: the available capacity of the electronic device's internal memory, the power status of the electronic device, the resources of an application running in the electronic device, or the resources of an artificial intelligence model running in the electronic device.

5. The electronic device as claimed in claim 1, wherein, The first information includes weights equal to or greater than a second threshold from the information obtained by pruning weights less than a first threshold, and the indices corresponding to the weights equal to or greater than the second threshold. The second information includes the weights of the data obtained through pruning that are less than a second threshold, and the indices corresponding to the weights that are less than the second threshold.

6. The electronic device as claimed in claim 5, wherein, The processor is configured to: The first information is changed by performing the first training based on the first information. The second information is changed by performing a second training based on the first and second information, and The first weight matrix can be obtained by loading the first changed information, or the second weight matrix can be obtained by loading the first changed information and the second changed information.

7. The electronic device as claimed in claim 1, wherein, The processor is configured to obtain a first weight matrix by erasing additional weights corresponding to the second index from the third information based on weights included in the third information corresponding to the second weight matrix, a third index corresponding to the weights, and a second index.

8. The electronic device as claimed in claim 7, wherein, The processor is configured to: The position of the additional weights in the second weight matrix is ​​determined based on the second and third indices. The additional weights are erased from the third information based on their positions, and The first weight matrix is ​​obtained by modifying the indices of the weights whose positions in the third information are adjacent to the additional weights that were previously removed and erased.

9. A control method for an electronic device storing first information and second information, the control method comprising: Obtain resource information of the electronic device, the resource information indicating the currently available resources of the electronic device. Based on the resource information of the electronic device, a first weight matrix is ​​obtained by loading first information, or a second weight matrix is ​​obtained by loading first information and second information. The first information includes weights associated with the first weight matrix and a first index corresponding to the weights. The second information includes additional weights for obtaining the second weight matrix and a second index corresponding to the additional weights. Specifically, when obtaining the second weight matrix, the second weight matrix is ​​obtained by combining the additional weights with the weights included in the first information based on the first index and the second index.

10. The control method as described in claim 9, further comprising: Output data is obtained by feeding input data into an artificial intelligence model that uses either a first weight matrix or a second weight matrix, and Based on the alteration of the resource information of the electronic device, a second weight matrix is ​​obtained based on the first weight matrix, or a first weight matrix is ​​obtained based on the second weight matrix.

11. The control method as described in claim 9, wherein, When obtaining the second weight matrix, the position of the weight corresponding to the first index in the second weight matrix is ​​determined based on the first index. The position of the additional weight corresponding to the second index in the second weight matrix is ​​determined based on the second index. Based on the position of the weight corresponding to the first index and the position of the weight corresponding to the second index, change at least one of the first index or the second index, and The second weight matrix is ​​obtained based on the position of the weights, the position of the additional weights, and the changed index.

Images