Electronic device and control method thereof

Abstract

An electronic device is provided. The electronic device includes a memory configured to store one or more instructions; and a processor configured to input input data to an artificial intelligence model including multiple layers by executing the instructions to obtain output data, the artificial intelligence model being configured to output the output data based on operations through the multiple layers, and the processor being configured to: encode operation data output from one of the multiple layers, store the encoded operation data in the memory, obtain recovery data corresponding to the operation data by decoding the encoded operation data stored in the memory, and provide the obtained recovery data to another layer from the multiple layers.

Description

Technical Field

[0001] The present invention relates to an electronic device and a control method thereof. More specifically, the present invention relates to an electronic device operating based on artificial intelligence technology and a control method thereof. Background Art

[0002] Recently, artificial intelligence (AI) systems have been developed that achieve human-level intelligence. Unlike existing rule-based intelligent systems, AI systems rely on machines to learn and make decisions independently, and are being used in various fields, such as speech recognition, image recognition, and future predictions. In particular, AI systems based on deep learning have recently been developed that can solve given problems.

[0003] At the same time, in order to implement artificial intelligence technology through an artificial intelligence model, it is necessary to store operation data of operation output based on the parameters of the artificial intelligence model and input data in a memory.

[0004] Here, the parameters of the AI ​​model are the weights of each layer of the AI ​​model, and since they are fixed values ​​after learning, they can be called static data. In addition, the operational data output based on the operation of the input data is the value of each subsequent output of the AI ​​model, and since it is a value determined by the input data, it can be called dynamic data.

[0005] Meanwhile, technology for compressing the above-mentioned fixed data by a method such as quantization has been developed, but technology for compression of dynamic data has not yet been developed.

[0006] The above information is provided as background information only to assist in understanding the present disclosure. No determination has been made, and no assertion is made, as to whether any of the above may be applied as prior art with respect to the present disclosure. Summary of the Invention

[0007]

Technical Issues

[0008] Aspects of the present invention are to address at least the above-mentioned problems and / or disadvantages and to provide at least the advantages described below. Accordingly, one aspect of the present invention is to provide an apparatus and method for an electronic device operating based on artificial intelligence technology and control methods thereof.

[0009] Additional aspects will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the presented embodiments.

[0010] According to one aspect of the present invention, an electronic device is provided. The electronic device includes: a memory configured to store one or more instructions; and a processor configured to input input data to an artificial intelligence model including multiple layers by executing the instructions to obtain output data, wherein the artificial intelligence model is configured to output the output data based on operations through the multiple layers, and the processor is configured to: encode operation data output from one of the multiple layers, store the encoded operation data in the memory, obtain recovery data corresponding to the operation data by decoding the encoded operation data stored in the memory, and provide the obtained recovery data to another layer from the multiple layers.

[0011] The multiple layers can be configured to include a first layer, a second layer, and a first hidden layer and a second hidden layer between the first layer and the second layer, wherein the artificial intelligence model is configured to learn weights of the first hidden layer and the second hidden layer to generate a second hidden layer for reconstructing the first layer, and wherein the processor is further configured to encode operation data based on the weights of the first hidden layer and decode the encoded operation data based on the weights of the second hidden layer.

[0012] The artificial intelligence model can also be configured to: output first output data based on an operation on the output data of the first layer and the weights of the first hidden layer, output second output data based on an operation on the first output data and the weights of the second hidden layer, and learn the weights of the first hidden layer and the second hidden layer so that the error between the output data of the first layer and the second output data of the second hidden layer is minimized.

[0013] The first hidden layer may have a dimension that is relatively lower than a dimension of the first layer, and wherein the second hidden layer has a dimension that is the same as the dimension of the first layer.

[0014] The artificial intelligence model may also be configured to obtain weights of the first hidden layer and the second hidden layer by learning the entire system including the plurality of layers and the first hidden layer and the second hidden layer.

[0015] The artificial intelligence model can also be configured as: based on adding the first hidden layer and the second hidden layer at the same time as the learning of multiple layers is completed, the multiple layers do not include the first hidden layer and the second hidden layer, and while fixing the weights of the multiple layers, the weights of the first hidden layer and the second hidden layer are obtained by learning the entire system including the multiple layers and the first hidden layer and the second hidden layer.

[0016] The multiple layers may include a first layer, a second layer, a third layer, a first hidden layer and a second hidden layer between the first layer and the second layer, and a third hidden layer and a fourth hidden layer between the second layer and the third layer, wherein the artificial intelligence model is further configured to: learn the weights of the first hidden layer and the second hidden layer to generate a second hidden layer for reconstructing the first layer, and learn the weights of the third hidden layer and the fourth hidden layer to generate a fourth hidden layer for reconstructing the second layer, and learn so that the first hidden layer and the third hidden layer have the same weights, and the second hidden layer and the fourth hidden layer have the same weights.

[0017] The processor may also be configured to: perform encoding of input data of the first hidden layer and encoding of input data of the third encoder through an encoder, and perform decoding of input data of the second decoder and decoding of input data of the fourth decoder through a decoder.

[0018] According to another aspect of the present invention, a method for controlling an electronic device is provided. The method includes encoding operation data output from one of multiple layers of an artificial intelligence model and storing the encoded operation data in a memory; decoding the encoded operation data stored in the memory to obtain restoration data corresponding to the operation data; and providing the obtained restoration data to another layer from the multiple layers.

[0019] The multiple layers may include a first layer, a second layer, and a first hidden layer and a second hidden layer between the first layer and the second layer, wherein the artificial intelligence model is configured to learn weights of the first hidden layer and the second hidden layer to generate a second hidden layer that reconstructs the first layer, wherein encoding includes encoding operation data based on the weights of the first hidden layer, and wherein decoding includes decoding the encoded operation data based on the weights of the second hidden layer.

[0020] The artificial intelligence model can also be configured to: output first output data based on an operation on the output data of the first layer and the weights of the first hidden layer, output second output data based on an operation on the first output data and the weights of the second hidden layer, and learn the weights of the first hidden layer and the second hidden layer so that the error between the output data of the first layer and the second output data of the second hidden layer is minimized.

[0021] The first hidden layer may have a dimension that is relatively lower than a dimension of the first layer, and wherein the second hidden layer has a dimension that is the same as the dimension of the first layer.

[0022] The artificial intelligence model may also be configured to obtain weights of the first hidden layer and the second hidden layer by learning the entire system including the plurality of layers and the first hidden layer and the second hidden layer.

[0023] The artificial intelligence model can also be configured as: based on adding the first hidden layer and the second hidden layer at the same time as the learning of multiple layers is completed, the multiple layers do not include the first hidden layer and the second hidden layer, and while fixing the weights of the multiple layers, the weights of the first hidden layer and the second hidden layer are obtained by learning the entire system including the multiple layers and the first hidden layer and the second hidden layer.

[0024] The multiple layers may include a first layer, a second layer, a third layer, a first hidden layer and a second hidden layer between the first layer and the second layer, and a third hidden layer and a fourth hidden layer between the second layer and the third layer, wherein the artificial intelligence model is further configured to: learn the weights of the first hidden layer and the second hidden layer to generate a second hidden layer for reconstructing the first layer, and learn the weights of the third hidden layer and the fourth hidden layer to generate a fourth hidden layer for reconstructing the second layer, and learn so that the first hidden layer and the third hidden layer have the same weights, and the second hidden layer and the fourth hidden layer have the same weights.

[0025] The control method may further include encoding input data of the first hidden layer and encoding input data of the third encoder through one encoder, and decoding input data of the second decoder and decoding input data of the fourth decoder through one decoder.

[0026] According to an embodiment, a computer-readable recording medium storing at least one instruction runs an instruction to cause an electronic device to encode operation data output from one layer of multiple layers of an artificial intelligence model, store the encoded operation data in a memory, obtain restored data corresponding to the operation data by decoding the encoded operation data stored in the memory, and provide the obtained restored data to another layer from among the multiple layers.

[0027] The computer-encoded operational data may be encoded in a vector having one dimension less than the vector comprising the operational data.

[0028] The vector may include the operation data including the same number of dimensions as the vector including the reduction data.

[0029]

Technical solution

[0030] The present invention aims to provide an electronic device capable of compressing and storing operational data, that is, an electronic device of dynamic data output by each layer of an artificial intelligence model, and a control method thereof.

[0031] Beneficial effects

[0032] According to the various embodiments described above, artificial intelligence technology can be effectively implemented even in mobile devices with limited memory through compression of dynamic data. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The above and other aspects, features and advantages of certain embodiments of the present invention will become more apparent from the following description taken in conjunction with the accompanying drawings, in which:

[0034] Figure 1 is a block diagram for explaining an electronic device according to an embodiment of the present invention;

[0035] Figure 2 is a detailed block diagram for explaining an electronic device according to an embodiment of the present invention;

[0036] Figure 3 is a view for explaining an embodiment of generating output data using encoded data according to an embodiment of the present invention;

[0037] Figure 4 is a diagram for explaining a learning method of an artificial intelligence model according to an embodiment of the present invention;

[0038] Figure 5 is a diagram for explaining multiple encodings and multiple decodings according to an embodiment of the present invention;

[0039] Figure 6 is a detailed block diagram for explaining an electronic device according to an embodiment of the present invention; and

[0040] Figure 7 is a flowchart for explaining a control method of an electronic device according to an embodiment of the present invention.

[0041] Throughout the drawings, like reference numerals will be understood to refer to like parts, components, and structures. DETAILED DESCRIPTION

[0042] The following description is provided with reference to the accompanying drawings to facilitate a more comprehensive understanding of the various embodiments of the present invention as defined by the claims and their equivalents. It includes numerous specific details to aid understanding, but these are to be regarded as exemplary only. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the various embodiments described herein without departing from the scope and spirit of the present invention. Furthermore, descriptions of well-known functions and configurations may be omitted for clarity and brevity.

[0043] The terms and expressions used in the following description and claims are not limited to the bibliographical meanings, but are merely used by the inventor to enable a clear and consistent understanding of the invention. Therefore, it should be clear to those skilled in the art that the following description of various embodiments of the invention is for illustration purposes only and is not intended to limit the invention as defined by the appended claims and their equivalents.

[0044] It should be understood that the singular forms "a," "an," and "the" include plural referents unless the context clearly dictates otherwise. Thus, for example, reference to "a component surface" includes reference to one or more of such surfaces.

[0045] Furthermore, if it is determined that a detailed explanation of related known functions or configurations may unnecessarily obscure the gist of the present invention when describing the embodiments, the detailed explanation will be abbreviated or omitted.

[0046] Furthermore, embodiments of the present invention will be described in detail with reference to the accompanying drawings and the contents described in the drawings, but the present invention is not limited thereto.

[0047] Hereinafter, the present disclosure will be described in detail with reference to the accompanying drawings.

[0048] The present invention aims to provide an electronic device capable of compressing and storing operational data, that is, an electronic device for dynamic data output by each layer of an artificial intelligence model, and a control method thereof.

[0049] Figure 1 is a block diagram for explaining an electronic device according to an embodiment of the present invention.

[0050] refer to Figure 1 , the electronic device 100 according to an embodiment includes a memory 110 and a processor 120 .

[0051] The electronic device 100 according to the embodiment is a device that uses an artificial intelligence model to obtain output data about input data, and the electronic device 100 may be, for example, a desktop personal computer (PC), a notebook PC, a smartphone, a tablet PC, a server, etc. Alternatively, the electronic device 100 may be the system itself in which the cloud computing environment is established. However, the electronic device 100 is not limited thereto and may be any device that can perform operations using an artificial intelligence model.

[0052] The memory 110 is provided separately from the processor 120 and can be implemented as a hard disk, a non-volatile memory, a volatile memory, etc. Here, the non-volatile memory can be a one-time programmable read-only memory (OTPROM), a programmable ROM (PROM), an erasable programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), a mask ROM, a flash ROM, etc., and the volatile memory can be a dynamic random access memory (DRAM), a static RAM (SRAM), or a synchronous dynamic RAM (SDRAM), etc. The memory 110 can be implemented as at least one of a ROM (not shown) or a RAM (not shown) in the processor 120.

[0053] The memory 110 is accessed by the processor 120 , and reading / recording / modification / deletion / update of data, etc. may be performed by the processor 120 .

[0054] The memory 110 may store an artificial intelligence model, which may be a model trained by an artificial intelligence algorithm.

[0055] The artificial intelligence model can be composed of multiple layers. Here, each layer can include one or more nodes (or neurons), and each node can be assigned at least one weight.

[0056] As such, the artificial intelligence model can be a neural network-based model. For example, the artificial intelligence model can be a recurrent neural network (RNN)-based model. Here, RNN refers to a cyclic neural network and is a deep learning model for learning data that changes over time, such as time series data.

[0057] However, the artificial intelligence model is not limited thereto, and the artificial intelligence model may be a model based on a variety of networks, such as a convolutional neural network (CNN), a deep neural network (DNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), etc. In addition, the memory 110 may store a model generated based on rules rather than a model trained by an artificial intelligence algorithm, and the model stored in the memory 110 is not particularly limited.

[0058] The processor 120 controls the overall operation of the electronic device 100. To this end, the processor 120 may be composed of one or more processors. Here, the one or more processors may be general-purpose processors, such as a central processing unit (CPU), an application processor (AP), or a communication processor (CP). The one or more processors may also be processors dedicated to graphics, such as a graphics processing unit (GPU), or processors dedicated to artificial intelligence, such as a neural network processing unit (NPU).

[0059] The processor 120 can perform the operations of the electronic device 100 according to multiple embodiments of the present invention by running at least one instruction stored in the memory 110. For example, the processor 120 can obtain output data about input data using an artificial intelligence model including multiple layers by running at least one instruction. Here, the input data can be text, images, user voice, etc. For example, the text can be text input through an input unit (not shown) such as a keyboard, touchpad, etc. of the electronic device 100, and the image can be an image taken by a camera of the electronic device 100. The user voice can be a user voice input through a microphone of the electronic device 100. However, this is only an example, and the processor 120 can obtain input data in a variety of ways. For example, the processor 120 can receive an image from an external device such as a server, or can receive a user voice from an external device such as a remote control. In addition, the above-mentioned input data can be, for example, various types of data of a predetermined time period, such as weather data, stock data, etc.

[0060] At the same time, the output data may vary depending on the type of input data and / or the AI ​​model. In other words, the output data may differ depending on which input data is fed into which AI model. For example, if the AI ​​model of the present invention is a model for language translation, processor 120 may obtain output data in a second language for input data expressed in a first language. Alternatively, if the AI ​​model of the present invention is a model for image analysis, processor 120 may input an image as input data to the AI ​​model and obtain information about objects detected from the corresponding image as output data. For example, if an image of a puppy is input as input data, processor 120 may obtain information that the corresponding image is an image of a puppy as output data. If the AI ​​model of the present invention is a model for image analysis, processor 120 may input an image as input data to the AI ​​model and obtain text representing the corresponding image as words as output data. For example, if an image of a boy holding an apple is input as input data, processor 120 may obtain the text "Boy picking apples" as output data. If the artificial intelligence model of the present invention is a model for speech recognition, the processor 120 can input the user's speech as input data and obtain text corresponding to the user's speech as output data. At the same time, the above output data is an example, and the type of output data of the present invention is not limited thereto.

[0061] When input data is input, the processor 120 can represent the input data as a vector (matrix or tensor). Here, the method of representing the input data as a vector (matrix or tensor) can vary according to the type of input data. For example, if text (or text converted from user voice) is input data, the processor 120 can represent the text as a vector by one hot encoding, or represent the text as a vector by word embedding. Here, one hot encoding is a method in which the value of only the index of a specific word is represented as 1 and the value of the remaining index is represented as 0, and word embedding is a method in which words are represented as real numbers in the dimension (e.g., 128 dimensions) of a vector set by the user. As examples of word embedding methods, Word2Vec, FastText, Glove, etc. can be used. At the same time, if an image is input as input data, the processor 120 can represent each pixel of the image as a matrix. For example, processor 120 may represent each pixel of an image as a value of 0 to 255 for each of red, green, and blue (RGB) colors, or represent the image as a matrix having values ​​obtained by dividing the values ​​represented as 0 to 255 by a preset number (e.g., 255).

[0062] The processor 120 may use an artificial intelligence model to perform operations on input data represented as vectors (matrices or tensors).

[0063] For example, if the artificial intelligence model consists of an input layer, a hidden layer, and an output layer, the input layer may include information about the input data represented as a vector (matrix or tensor). In this case, the processor 120 can output the operation data of the input data by calculating the weights between the input layer and the hidden layer read from the memory 110 in the vector (or matrix or tensor) of the input data. Here, bias can be further considered, but for ease of explanation, bias will be omitted in the description. In addition, the processor 120 can perform operations by using the operation data of the input layer as the input of the hidden layer. Specifically, the processor 120 can output the operation data of the input layer and the operation data of the hidden layer by performing operations based on the weights between the hidden layer and the output layer read from the memory 110. The processor 120 can obtain output data using the operation data of the hidden layer as the input of the output layer. Specifically, the processor 120 can output output data by applying a predetermined function (e.g., a softmax function) to the operation data of the hidden layer. Here, as described above, if the artificial intelligence model is a model for language translation, the output data may be text in a language different from the input text input as input data, and if the artificial intelligence model is a model for image analysis, the output data may be data including information about objects included in the image. However, the output data is not limited thereto. At the same time, if the artificial intelligence model of the present invention is a model including multiple hidden layers, the operation data output in the previous hidden layer may be input data for the next hidden layer.

[0064] The processor 120 can store the operation data of each layer of the artificial intelligence model in the memory 110. This is because in the stage of training the artificial intelligence model, in the stage of backpropagation, the operation data for updating the weights of each layer is needed to minimize the error of the artificial intelligence model, and in the stage of reasoning after training the artificial intelligence model, the operation data of the previous layer is needed to perform the operation of the next layer. For example, if the artificial intelligence model of the present invention is a recurrent neural network (RNN), the processor 120 can store the operation data output based on the input data of the previous time point in the memory 110, so as to use the operation data in the process of outputting the operation data of the input data of the next time point. In addition, if the artificial intelligence model of the present invention is a convolutional neural network (CNN), the processor 120 can store the operation data (i.e., feature map) generated by performing convolution operation processing on the previous layer in the memory 110, so as to use the operation data for the operation of the next layer.

[0065] Meanwhile, the operational data is data determined based on the data input to each layer as described above, and thus, may be referred to as dynamic data, intermediate data, or activation.

[0066] Meanwhile, when the operation data is stored in the memory 110 as described above, there may be a problem of exceeding the capacity of the memory in a mobile device such as a smartphone having a limited memory.

[0067] To solve such a problem, the processor 120 may encode (or compress) the operation data and store it in the memory 110, which will be referred to as Figure 2 Provide explanation.

[0068] Figure 2 is a detailed block diagram for explaining an electronic device according to an embodiment of the present invention.

[0069] refer to Figure 2 , the electronic device 100 according to the embodiment may include a memory 110, an encoder 130, a decoder 140, and a processor 120. Hereinafter, overlapping descriptions will be omitted or abbreviated.

[0070] The processor 120 is connected to one or more components constituting the electronic device 100 and controls the overall operation of the electronic device 100. Specifically, the processor 120 is connected to at least one of the memory 110, the encoder 130, or the decoder 140 to control the operation of each component.

[0071] The processor 120 can obtain input data. For example, the processor 120 can obtain not only text input through an input unit (not shown) such as a keyboard or touchpad of the electronic device 100 as input data, but also images captured by a camera of the electronic device 100 as input data. In addition, the processor 120 can obtain user voice input through a microphone of the electronic device 100 as input data.

[0072] The processor 120 can represent the input data as a vector (a matrix or tensor according to an embodiment). Here, depending on the type of input data, the method of representing the input data as a vector (matrix or tensor) can be different. For example, if the text of "I am a boy" is obtained as input data, the processor 120 can represent "I" as [1, 0, 0, 0], "am" as [0, 1, 0, 0], "a" as [0, 0, 0, 1], and "boy" as [0, 0, 0, 1] by single-hot encoding. Alternatively, the processor 120 can represent "I" as [0.1, 4.2, 1.5, 2.8], "am" as [1.0, 3.1, 2.5, 1.1], "a" as [0.3, 2.1, 0.9, 1.1], and "boy" as [0.7, 1.7, 0.5, 0.2] by word embedding. If an image is obtained as input data, the processor 120 may represent the image as a matrix in an M×N form based on the RGB value of each pixel (here, M and N may not only be different integers but also the same integer).

[0073] The processor 120 can use the artificial intelligence model to perform operations on input data represented in a vector (matrix or tensor). For example, if the artificial intelligence model consists of an input layer, a hidden layer, and an output layer, the input layer may include information about the input data represented in a vector (matrix or tensor). In this case, the processor 120 can output the operation data of the input layer by calculating the weights between the input layer and the hidden layer read from the memory 110.

[0074] In this case, the operation may be a deep learning operation in the form of y=M*x (where M refers to a vector, matrix, or tensor representing weights read from the memory 110, x refers to a vector, matrix, or tensor representing input data, and y indicates a vector, matrix, or tensor representing operation data). For example, if the input data is [x1; x2] and the weights read from the memory 110 are [w11, w12; w21, w22], the operation data may be [w11*x1+w12*x2; w21*x1+w22*x2]. To this end, the processor 120 may include a processing element (PE) that performs the operation on the data and may be connected to an interface of the memory 110.

[0075] The processor 120 may perform an operation of storing operation data in the memory 110 .

[0076] Specifically, the processor 120 may encode the operation data and store it in the memory 110. As described above, the operation data of the artificial intelligence model may be represented as a vector, a matrix, or a tensor, and such a vector, matrix, or tensor is the subject of the encoding of the present invention.

[0077] To this end, the processor 120 may be connected to an encoder 130, such as Figure 2 The processor 120 may transmit operation data generated based on the deep learning operation to the encoder 130 .

[0078] The encoder 130 may encode the operation data output by the processor 120. Here, encoding refers to compression of the operation data, and the encoder 130 may convert the operation data into data smaller than the operation data through encoding, or represent the operation data as data smaller than the operation data.

[0079] According to an embodiment, the operation data can be converted into or represented as data having a dimension lower than the dimension of the operation data by encoding. In other words, if the operation data is an n-dimensional vector (matrix or tensor), the encoder 130 can convert the operation data into a vector (matrix or tensor) having a dimension less than n, or represent the operation data as a vector (matrix or tensor) having a dimension less than n. For example, if the operation data output by the above-mentioned deep learning is represented as a four-dimensional vector such as [0.7, 1.7, 0.5, 0.2], the encoder 130 can represent the operation data of [0.7, 1.7, 0.5, 0.2] as a three-dimensional vector such as [0.5, 1.5, 0.4].

[0080] To this end, when receiving operation data from the processor 120 , the encoder 130 may read the weight used to encode the operation data from the memory 110 .

[0081] Specifically, when the operation data of layer n is received from the processor 120, the encoder 130 can read the weight corresponding to layer n from the memory 110. Here, the weight corresponding to layer n means the weight learned for encoding the operation data of layer n. Figure 4 Describe the learning of the artificial intelligence model of the present invention.

[0082] The encoder 130 may encode the operation data of the layer n by performing a deep learning operation based on the operation data received from the processor 120 and the weights read from the memory 110. In other words, the encoder 130 may convert or represent the operation data of the layer n represented as an n-dimensional vector (matrix or tensor) into a vector (matrix or tensor) having a dimension less than n.

[0083] The encoder 130 may then store the encoded data in the memory 110. To this end, the encoder 130 may be connected to the memory 110, such as Figure 2 shown.

[0084] Meanwhile, it has been described above that the operation data based on the input data and the weights of the hidden layer are encoded, but this is only an example. If the artificial intelligence model of the present invention includes multiple hidden layers, the processor 120 can encode the operation data based on the output data of the first hidden layer (corresponding to the input data of the second hidden layer) and the weights of the second hidden layer through the encoder 130. The operation data to be encoded can be not only the output data of the previous layer, but also the data of the activation function (e.g., ReLu, sigmoid) of the current layer applied to the output data of the previous layer.

[0085] The decoder 140 may decode the encoded data stored in the memory 110 .

[0086] Decoding means restoring the encoded data, and the decoder 140 can restore the encoded data to data corresponding to the data before encoding. Here, data corresponding to the data before encoding means data similar to the data before encoding, and may not be completely the same as the data before encoding, but can be regarded as data that can be used to reconstruct the data before encoding.

[0087] According to an embodiment, the encoded data stored in the memory 110 may be data encoded by the encoder 130, that is, the operation data output by the processor 120 is converted or represented as data having a low dimension by the encoder 130. In this case, the decoder 140 can obtain restored data corresponding to the above-mentioned operation data by decoding the encoded data stored in the memory 110.

[0088] Here, the restoration data corresponding to the operation data is data that is similar to the operation data output by the processor 120 and may not be completely identical to the operation data, but may be data that can be regarded as data for reconstructing the operation data.

[0089] According to an embodiment, the encoded data may be converted or represented by decoding as data having a dimension higher than that of the encoded data. In other words, if the encoded data is an n-dimensional matrix, the decoder 140 may convert or represent the encoded data into or represent it as a matrix having a dimension greater than n. For example, if the data encoded by the encoder 130 is represented as a three-dimensional vector such as [0.5, 1.5, 0.4], the decoder 140 may represent the encoded data of [0.5, 1.5, 0.4] as a four-dimensional vector such as [0.7, 1.7, 0.5, 0.2].

[0090] To this end, the decoder 140 may read the encoded data stored in the memory 110 and read the weights for decoding the encoded data from the memory 110. In addition, the decoder 140 may decode the encoded data by performing a deep learning operation on the encoded data and weights read from the memory 110. In other words, the decoder 140 may restore the encoded data represented in an n-dimensional matrix to a matrix having a dimension higher than n. To this end, the decoder 140 may be connected to the memory 110, such as Figure 2 shown.

[0091] Thereafter, the decoder 140 may transmit the decoded data obtained by decoding to the processor 120. To this end, the decoder 140 may be connected to the processor 120, such as Figure 2 shown.

[0092] The processor 120 may then perform operations using the decoded data received from the decoder 140, which will then be referenced. Figure 3 describe.

[0093] Therefore, the present invention can efficiently implement an artificial intelligence model in a memory with limited capacity by encoding operation data and storing it in the memory.

[0094] At the same time, Figure 2 1 , the memory 110, the processor 120, the encoder 130, and the decoder 140 are shown as separate components, but this is merely an example. For example, the encoder 130 and the decoder 140 may be included in the memory 110. In this case, the encoder 130 may encode the operating data output by the processor 120 within the memory, and may store the encoded data in the memory 110. The decoder 140 may decode the data encoded by the encoder 130 within the memory 110. Therefore, the present invention can remove or reduce the power consumption caused by the transmission / reception of data between the encoder 130 and the memory 110, as well as the power consumption caused by the transmission / reception of data between the decoder 140 and the memory 110.

[0095] In addition, the encoder 130 and the decoder 140 may be included in the processor 120. In this case, the encoder 130 may encode the operation data generated by the processor 120 within the processor, and the decoder 140 may decode the data encoded by the encoder 130 within the processor 120. Therefore, the present invention can remove or reduce power consumption caused by data transmission / reception between the encoder 130 and the processor 120 and power consumption caused by data transmission / reception between the decoder 140 and the processor 120.

[0096] In addition, the processor 120, the encoder 130, and the decoder 140 may be included in the memory 110. In this case, the processor 120 may perform a series of operations for obtaining output data from input data in the memory 110, and store operation data generated as a result of the operations in the memory 110. The encoder 130 may perform encoding of the operation data generated by the processor 120 in the memory 110, and the decoder 140 may perform decoding of data encoded by the encoder 130 in the memory 110.

[0097] Figure 3 is a view for explaining an embodiment of generating output data using encoded data according to an embodiment of the present invention.

[0098] The artificial intelligence model according to an embodiment may include an input layer for inputting input data, an output layer for outputting output data, and a plurality of hidden layers located between the input layer and the output layer and outputting operation data based on deep learning operations.

[0099] Specifically, refer to Figure 3 The artificial intelligence model according to an embodiment may further include a first hidden layer 320 and a second hidden layer 330. The first hidden layer 320 includes encoded data encoding operational data output by a specific layer 310 (the specific layer may be one of multiple hidden layers or an input layer), and the second hidden layer 330 includes decoded data decoding the encoded data of the first hidden layer 320. Here, the first hidden layer 320 may be referred to as an encoding layer, and the second hidden layer 330 may be referred to as a decoding layer. The encoded data included in the first hidden layer 320 may be stored in the memory 110 as described above. If the artificial intelligence model according to an embodiment is implemented as a convolutional neural network (CNN), the processor 120 may encode the operational data (i.e., feature map) generated by performing a convolution operation on a previous layer (not shown) through the encoder 130 and store it in the memory 110. The processor 120 may obtain decoded data by restoring the encoded data for the operation of the next layer 340 and perform operations based on the decoded data and the weights of the next layer 340.

[0100] Alternatively, if the artificial intelligence model according to the embodiment is a recurrent neural network (RNN), the processor 120 may encode the operation data output based on the input data at the previous time point (t-1) through the encoder 130 and store it in the memory 110. In the operation of outputting the operation data of the input data at the current time point (t), the processor 120 may output the operation data by considering decoded data generated by restoring the encoded data stored in the memory together with the input data at the current time point.

[0101] At the same time, despite Figure 3 One encoding layer and one decoding layer are shown, but there may be multiple encoding layers and decoding layers depending on the embodiment.

[0102] Figure 4 It is a view for explaining a learning method of an artificial intelligence model according to an embodiment of the present invention.

[0103] refer to Figure 4 , the artificial intelligence model according to the embodiment may include multiple layers. The artificial intelligence model may also include a first hidden layer (not shown) for encoding the operation data and a second hidden layer (not shown) for decoding the encoded data. Here, the first hidden layer may be relatively lower than the dimension of layer n, and the second hidden layer may be the same as the dimension of layer n.

[0104] For example, an artificial intelligence model according to an embodiment may include a first hidden layer for encoding operation data of layer n between layer n and layer n+1, and a second hidden layer for decoding the encoded data of the first hidden layer. Thus, the present invention is characterized in that encoding and decoding are performed between some of the multiple layers constituting the artificial intelligence model.

[0105] At the same time, in the artificial intelligence model according to the embodiment, multiple encoding and decoding can be performed between multiple layers. In the following, for ease of explanation, it is assumed that the artificial intelligence model includes a first hidden layer for encoding and a second hidden layer for decoding.

[0106] An artificial intelligence model including a first hidden layer and a second hidden layer between layer n and layer n+1 can learn weights of the first hidden layer and the second hidden layer to generate a second hidden layer to reconstruct layer n.

[0107] Specifically, the artificial intelligence model can learn the weights of the first hidden layer and the second hidden layer so as to reconstruct the data of the nth layer into a layer having a number of nodes less than the number of nodes (or neurons) of layer n.

[0108] For example, if a first hidden layer and a second hidden layer are added between layer n and layer n+1, the artificial intelligence model can output the output data of the first hidden layer based on the operation data of layer n (i.e., the output data of layer n) and the operation of the weights of the first hidden layer, output the output data of the first hidden layer, and output the output data of the second hidden layer based on the operation of the output data of the first hidden layer and the weights of the second hidden layer. In addition, the artificial intelligence model can learn the weights of the first hidden layer and the second hidden layer so that the error between the output data of layer n and the output data of the second hidden layer is minimized.

[0109] According to an embodiment, the artificial intelligence of the present invention may have a structure similar to an auto-encoder. An auto-encoder is an artificial intelligence model that performs learning to output data similar to input data, and for example, if encoding and decoding are performed between layer n and layer n+1, the first hidden layer and the second hidden layer of the present invention can be trained to output data similar to the data output from layer n, which is similar to an auto-encoder. However, the auto-encoder is different from the artificial intelligence model of the present invention in that the auto-encoder itself exists as an artificial intelligence model, but the encoding and decoding of the present invention are performed only between some layers of the multiple layers of the artificial intelligence model.

[0110] At the same time, the artificial intelligence model can be learned by learning the entire system including the first hidden layer and the second hidden layer. In this case, the artificial intelligence model can perform learning so that the error of the entire system including the first hidden layer and the second hidden layer is minimized. Here, the learning can be performed by general forward propagation and backpropagation, and therefore, a detailed description is omitted.

[0111] As described above, by learning the entire system including multiple layers and the first hidden layer and the second hidden layer added between some of the multiple layers, the artificial intelligence model of the present invention can obtain the weights of multiple layers and the weights of the first hidden layer and the second hidden layer at one time.

[0112] While this is merely an example, the present invention can independently learn the first hidden layer and the second hidden layer. For example, if the first hidden layer and the second hidden layer are added between layer n and layer n+1, the present invention can set the operating data of layer n as the input of the first hidden layer and the operating data of layer n as the output of the second hidden layer. Furthermore, by learning the weights of the first hidden layer and the second hidden layer to reconstruct the data of layer n, which is relatively smaller than the data size of layer n, the weights of the first hidden layer and the second hidden layer can be obtained.

[0113] Therefore, when receiving operation data from the processor 120, the encoder 130 can perform encoding using the received operation data based on the learned weights. In addition, the decoder 140 can perform decoding using the encoded data stored in the memory 110 based on the learned weights.

[0114] Meanwhile, if the first hidden layer and the second hidden layer are added while learning of multiple layers excluding the first hidden layer and the second hidden layer is completed, the artificial intelligence model according to the embodiment can further learn the weights of the first hidden layer and the second hidden layer.

[0115] In this case, the artificial intelligence model can fix the weights of the multiple layers it has learned. In other words, the artificial intelligence model can fix the weights of the multiple layers it has learned and perform learning so that the error of the entire system including the first hidden layer and the second hidden layer is minimized.

[0116] Therefore, depending on whether compression is required, the present invention can selectively add the first hidden layer and the second hidden layer to the artificial intelligence model.

[0117] Figure 5 It is a view for explaining multiple encodings and multiple decodings according to an embodiment of the present invention.

[0118] The artificial intelligence model according to an embodiment may perform multiple encodings and multiple decodings.

[0119] refer to Figure 5 , the artificial intelligence model may include a first hidden layer (not shown) and a second hidden layer (not shown) between the first layer and the second layer, and a third hidden layer (not shown) and a fourth hidden layer (not shown) between the third layer and the fourth layer. Here, the first hidden layer may be a layer for encoding the operation data of the first layer, and the second hidden layer may be a layer for decoding the encoded data of the first hidden layer. The third hidden layer may be a layer for encoding the operation data of the third layer, and the fourth hidden layer may be a layer for decoding the encoded data of the third hidden layer.

[0120] Each of the first to fourth hidden layers included in the artificial intelligence model can be trained to generate an output layer that reconstructs the previous layer. For example, in the above embodiment, the first and second hidden layers can be trained to reconstruct the first layer, and the third and fourth hidden layers can be trained to reconstruct the third layer.

[0121] In this case, the encoder 130 according to the embodiment may be implemented as a plurality of encoders. In other words, the electronic device 100 according to the embodiment may include a first encoder (not shown) for encoding the output data of the first layer using the weight of the first hidden layer and a second encoder (not shown) for encoding the output data of the third layer using the weight of the third hidden layer.

[0122] Similarly, the decoder 140 according to the embodiment may be implemented in a plurality of ways. In other words, the electronic device 100 according to the embodiment may include a first decoder (not shown) for decoding data encoded by a first encoder (not shown) using the weights of the second hidden layer and a second decoder (not shown) for decoding data encoded by a second encoder (not shown) using the weights of the fourth hidden layer.

[0123] This is to take into account that the characteristics of each layer may be different, and correspondingly, for each layer, the weights used for encoding and the weights used for decoding may be different. Therefore, the present invention can encode and decode a variety of data through multiple encoders and multiple decoders.

[0124] At the same time, the artificial intelligence model according to the embodiment can perform learning so that multiple hidden layers for encoding share weights and multiple hidden layers for decoding share weights.

[0125] Specifically, the artificial intelligence model can learn so that the first hidden layer and the third hidden layer have the same weight by sharing the weights of the first hidden layer and the third hidden layer, and the second and fourth hidden layers have the same weight by sharing the weights of the second hidden layer and the fourth hidden layer. In this case, in the operation of performing learning, the artificial intelligence model can update the values ​​of the weights of the first hidden layer and the third hidden layer so that the first hidden layer and the third hidden layer have the same weight, and update the values ​​of the weights of the second hidden layer and the fourth hidden layer so that the second hidden layer and the fourth hidden layer have the same weight.

[0126] Therefore, the electronic device 100 according to the embodiment may encode output data of different layers through one encoder 130. Similarly, the electronic device 100 according to the embodiment may decode data encoded in different layers through one decoder 140.

[0127] Therefore, compared with the case of including a plurality of encoders and a plurality of decoders, the present invention can be implemented in the electronic device 100 having a relatively small size. In addition, by sharing weights, the time required for learning can be reduced.

[0128] Figure 6 is a detailed block diagram for explaining an electronic device according to an embodiment of the present invention.

[0129] refer to Figure 6 , the electronic device 100 according to the embodiment may include a memory 110, an encoder 130, a decoder 140, an input unit 150, a communicator 160, a display 170, a speaker 180, and a processor 120. Hereinafter, overlapping descriptions will be omitted or abbreviated.

[0130] The input unit 150 can receive user commands. To this end, the input unit 150 may include a touch sensor, a (digital) pen sensor, a pressure sensor, or a key. The touch sensor may use at least one of a capacitive method, a resistive method, an infrared method, or an ultrasonic method. For example, the (digital) pen sensor may be part of a touch panel, or may include a separate piece of paper for identification. The key may be, for example, a physical button, an optical key, or a keypad. The input unit 150 may include a microphone. Here, the microphone may receive sounds emitted by the user.

[0131] The communicator 160 can transmit / receive various data by performing communication with an external device. Specifically, the communicator 160 can receive an artificial intelligence model by performing communication with an external device. Here, the artificial intelligence model can be not only a model including a layer for encoding and a layer for decoding, but also a model without a layer for encoding and a layer for decoding. In the latter case, the electronic device 100 can add a layer for encoding and a layer for decoding to the artificial intelligence model and train the artificial intelligence model including a layer for encoding and a layer for decoding.

[0132] In addition, the communicator 160 may receive various data for generating an artificial intelligence model by performing communication with an external device.

[0133] To this end, the communicator 160 may include a wireless communication chip, a Wi-Fi chip, a Bluetooth chip, etc.

[0134] The display 170 can display a variety of screens. Specifically, the display 170 can display a screen corresponding to the output data output by the artificial intelligence model. Here, the screen corresponding to the output data can be a message, image, etc. generated based on the output data.

[0135] The display 170 may be implemented using various display technologies, such as a liquid crystal display (LCD), an organic light emitting diode (OLED), an active matrix organic light emitting diode (AM-OLED), a liquid crystal on silicon (LCoS), a digital light processing (DLP), etc. In addition, the display 170 may be connected to at least one of the front region, the side region, or the rear region of the electronic device 100 in the form of a flexible display.

[0136] In addition, the display 170 may be implemented as a touch screen having a touch sensor.

[0137] The speaker 180 is configured to output various audio data, on which an audio processor (not shown) has performed various processing operations such as decoding, amplification, and noise filtering. In addition, the speaker 180 can output various alarm sounds or voice messages. According to an embodiment, when the learning of the artificial intelligence model is completed or the output data is output by the artificial intelligence model, the speaker 180 can output an alarm sound, etc.

[0138] Figure 7 is a flowchart for explaining a method for controlling an electronic device according to an embodiment of the present invention.

[0139] In operation S710, the electronic device 100 according to an embodiment may encode the operation data output from one of the multiple layers of the artificial intelligence model and store the encoded data in the memory. Specifically, the electronic device 100 may encode the operation data output from one of the multiple layers of the artificial intelligence model through an encoder and store the encoded data in the memory 110.

[0140] The electronic device 100 can obtain the restored data corresponding to the operation data by decoding the encoded data stored in the memory 110 (in operation S720). Specifically, the electronic device 100 can obtain the restored data corresponding to the operation data by decoding the encoded data through the decoder. Therefore, the present invention can effectively implement artificial intelligence technology through dynamic data compression, even in mobile devices with limited memory.

[0141] In operation S730, the electronic device 100 may provide the obtained restored data to another layer of the plurality of layers of the artificial intelligence model. Thus, the output layer of the artificial intelligence model may generate output data based on the operation data and the restored data of the previous layer.

[0142] According to the various embodiments described above, artificial intelligence technology can be effectively implemented even in mobile devices with limited memory through dynamic data compression.

[0143] Meanwhile, the above methods according to various embodiments can be implemented by simply upgrading the software / hardware of existing electronic devices.

[0144] Furthermore, the above-described various embodiments may be executed by an embedded server or an external server provided in the electronic device.

[0145] The control methods of the electronic device according to the above-mentioned embodiments can be implemented as programs and stored in various recording media. In other words, computer programs that are processed by various processors and execute the above-mentioned control methods can be stored in recording media and used therein.

[0146] Non-transitory computer-readable media refers to media that semi-permanently stores data and can be read by a device, but is not used to store data for a short period of time, such as registers, caches, and memories. Specifically, the various applications or programs mentioned above can be stored in non-transitory computer-readable media, such as compact discs (CDs), digital versatile discs (DVDs), hard disks, Blu-ray discs, USBs, memory cards, ROMs, etc., and provided therein.

[0147] While the invention has been shown and described with reference to several embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made without departing from the spirit and scope of the invention as defined by the appended claims and their equivalents.

Claims

1. An electronic device comprising: a memory configured to store one or more instructions; An artificial intelligence model comprising multiple layers, the multiple layers comprising: The first and second floors, and The first hidden layer and the second hidden layer between the first and second layers, wherein the artificial intelligence model is configured to operate on input data using the plurality of layers to generate output data, and to generate operation data during processing of each of the plurality of layers; and A processor configured to input input data into the artificial intelligence model by executing the instructions to obtain output data by: encoding the operational data output from the first layer using the weights of the first hidden layer of the artificial intelligence model, storing the encoded operation data in a memory, wherein the operation data output from the first layer is a matrix having entries representing outputs of an operation performed by the first layer on the input data, and the encoded operation data is a compressed version of the operation data, Obtaining restored data corresponding to the operation data by decoding the encoded operation data stored in a memory using the weights of a second hidden layer of the artificial intelligence model, wherein the restored data is a decompressed version of the encoded operation data; and providing the obtained recovery data to the second layer, The input data is text, image or speech.

2. The device according to claim 1, wherein The artificial intelligence model is also configured to: Output first output data based on the operation of the output data of the first layer and the weight of the first hidden layer, outputting second output data based on an operation of the first output data and the weights of the second hidden layer, and Minimize the error between the output data of the first layer and the second output data of the second hidden layer.

3. The device according to claim 2, in, The first hidden layer has a dimension lower than the dimension of the first layer, and The second hidden layer has the same dimensions as the first layer.

4. The device according to claim 1, wherein The plurality of layers include a third layer, and a third hidden layer and a fourth hidden layer between the second layer and the third layer, The processor is further configured to: performing encoding of input data of the first hidden layer and encoding of input data of the third encoder by an encoder, and Decoding of input data of the second decoder and decoding of input data of the fourth decoder are performed by one decoder.

5. A method for controlling an electronic device, the method comprising: processing input data using an artificial intelligence model, the artificial intelligence model comprising a plurality of layers, the plurality of layers comprising a first layer and a second layer, and a first hidden layer and a second hidden layer between the first layer and the second layer, wherein the artificial intelligence model is configured to operate on the input data using the plurality of layers to generate output data, and to generate operation data during processing of each of the plurality of layers; encoding operational data output from the first hidden layer using weights of the first hidden layer of the artificial intelligence model; storing the encoded operation data in a memory, wherein the operation data output from the first layer is a matrix having entries representing outputs of an operation performed by the first layer on input data, and the encoded operation data is a compressed version of the operation data; Obtaining recovered data corresponding to the operation data by decoding the encoded operation data stored in a memory using weights of a second hidden layer of the artificial intelligence model, wherein the recovered data is a decompressed version of the encoded operation data; as well as providing the obtained recovery data to the second layer, The input data is text, image or speech.

6. The control method according to claim 5, wherein: The artificial intelligence model is also configured to: Output first output data based on the operation of the output data of the first layer and the weight of the first hidden layer, outputting second output data based on an operation of the first output data and the weights of the second hidden layer, and Minimize the error between the output data of the first layer and the second output data of the second hidden layer.

7. The control method according to claim 6, in, The first hidden layer has a dimension lower than the dimension of the first layer, and The second hidden layer has the same dimensions as the first layer.

Images