Electronic device and control method thereof

By compressing and adjusting multiple AI models in electronic devices, using pruning, quantization, decomposition and knowledge distillation technologies, the problems of storage and performance limitations in smart devices are solved, and the efficient use of multiple AI models is achieved. Effect.

CN112424799BInactive Publication Date: 2024-11-19SAMSUNG ELECTRONICS CO LTD
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN201980047725.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2018-07-19
Filing Date
2019-07-18
Publication Date
2024-11-19
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

It is difficult for the prior art to efficiently use multiple AI models in smart devices, mainly due to storage capacity and performance limitations, resulting in poor recognition rate and user preference understanding of AI models.

Method used

By implementing the compression of multiple artificial intelligence models in electronic devices, using technologies such as pruning, quantization, decomposition and knowledge distillation, the performance and size of the model are dynamically adjusted according to the memory size and frequency of use, ensuring that the model operates efficiently in limited memory.

Benefits of technology

It realizes efficient use of multiple artificial intelligence models in limited memory, dynamically adjusts model performance and size, improves recognition rate and user preference understanding effect, and solves the problems of storage and performance limitations.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN112424799B_ABST
    Figure CN112424799B_ABST
Patent Text Reader

Abstract

A method for controlling an electronic device includes: storing multiple artificial intelligence models in a first memory; identifying the available memory size of the second memory based on receiving a control signal for loading a first artificial intelligence model among the multiple stored artificial intelligence models into a second memory; and based on the size of the first artificial intelligence model being greater than the available memory size of the second memory, obtaining a first compressed artificial intelligence model by compressing the first artificial intelligence model based on the available memory size of the second memory, and loading the first compressed artificial intelligence model into the second memory.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to an electronic device and a control method thereof, and more particularly, to an electronic device and a control method thereof for efficiently using a plurality of artificial intelligence models.

[0002] In addition, the present disclosure relates to an artificial intelligence (AI) system and its applications that use machine learning algorithms to simulate functions of the human brain, such as cognition and determination. Background Art

[0003] Artificial intelligence (AI) systems are computer systems that achieve human-level intelligence. Unlike the rule-based intelligent systems of the related art, artificial intelligence systems are systems in which machines learn, judge, and become intelligent. The more an artificial intelligence system is used, the higher the recognition rate and the better the understanding of user preferences. Therefore, the rule-based intelligent systems of the related art have been gradually replaced by artificial intelligence systems based on deep learning.

[0004] Artificial intelligence technology consists of machine learning (e.g., deep learning) and element technology using machine learning.

[0005] Machine learning is an algorithmic technology that classifies / trains the features of input data by itself. Elemental technology is a technology that uses machine learning algorithms such as deep learning to simulate functions of the human brain such as recognition and judgment, and includes language understanding, visual understanding, reasoning / prediction, knowledge representation, motion control, etc.

[0006] Artificial intelligence technology can be applied to various fields, examples of which are described below. Language understanding is a technology for recognizing and applying / processing human language / characters, including natural language processing, machine translation, dialogue systems, query response, speech recognition / synthesis, etc. Visual understanding is a technology for recognizing and processing objects as if perceived by humans, including object recognition, object tracking, image search, human recognition, scene understanding, spatial understanding, image enhancement, etc. Reasoning prediction is a technology for judging and logically reasoning and predicting information, including knowledge / probability-based reasoning, optimization prediction, preference-based planning, and recommendation. Knowledge representation is a technology for automating human experience information into knowledge data, including knowledge construction (data generation / classification) and knowledge management (data utilization). Motion control is a technology for controlling the autonomous motion of a device or object (e.g., the travel of a vehicle and the movement of a robot), including motion control (navigation, collision and travel), operation control (behavior control), etc.

[0007] Recently, various artificial intelligence models for various purposes can be run on a single device or application. However, the artificial intelligence technology of the related art requires a lot of time and a large storage capacity, so it is not feasible to use it efficiently in user terminal devices such as smart phones. Summary of the invention

[0008] Technical Solution

[0009] According to one aspect of the present disclosure, a method for controlling an electronic device is provided, the method comprising: storing multiple artificial intelligence models in a first memory; identifying the available memory size of the second memory based on receiving a control signal for loading a first artificial intelligence model among the multiple stored artificial intelligence models into a second memory; and based on the size of the first artificial intelligence model being greater than the available memory size of the second memory, obtaining a first compressed artificial intelligence model by compressing the first artificial intelligence model based on the available memory size of the second memory, and loading the first compressed artificial intelligence model into the second memory.

[0010] Loading may include: identifying whether the performance of the first compressed artificial intelligence model meets predetermined conditions; based on the performance of the first compressed artificial intelligence model meeting the predetermined conditions, loading the first compressed artificial intelligence model into a second memory; and based on the first compressed artificial intelligence model not meeting the predetermined conditions, obtaining a second compressed artificial intelligence model obtained by compressing the first artificial intelligence model using a method different from the first compressed artificial intelligence model, and loading the second compressed artificial intelligence model into a second memory.

[0011] The performance of the first compressed artificial intelligence model may be any one of or any combination of a processor share rate of the first compressed artificial intelligence model, a size of the first compressed artificial intelligence model, and an accuracy of the first compressed artificial intelligence model.

[0012] The method may further include: based on the fact that a plurality of compressed artificial intelligence models obtained by compressing the first artificial intelligence model using a plurality of methods do not satisfy the predetermined condition, displaying a message notifying that the predetermined condition is not satisfied.

[0013] The method may further include: based on the available memory size of the second memory being changed, obtaining a third compressed artificial intelligence model by compressing the first artificial intelligence model based on the changed available memory size, and loading the third compressed artificial intelligence model.

[0014] The method may further include: based on receiving a control signal for loading the second artificial intelligence model into the second memory, loading a compressed artificial intelligence model for the first artificial intelligence model obtained by compressing the first artificial intelligence model, and a compressed artificial intelligence model for the second artificial intelligence model obtained by compressing the second artificial intelligence model according to the available memory size of the second memory.

[0015] The method may further include: based on loading multiple artificial intelligence models into a second memory, identifying the number of times the multiple artificial intelligence models loaded in the second memory are used; and based on the identified number of times, loading a compressed artificial intelligence model for at least one of the multiple artificial intelligence models.

[0016] The method may further include displaying a user interface (UI) for receiving performance of at least one of the plurality of artificial intelligence models, wherein the first compressed artificial intelligence model may be an artificial intelligence model compressed based on the performance input through the UI.

[0017] Storing may further include storing a plurality of compressed artificial intelligence models for the first artificial intelligence model in a first memory.

[0018] The first artificial intelligence model can be compressed using any one of pruning, quantization, decomposition, and knowledge distillation, or any combination thereof.

[0019] According to one aspect of the present disclosure, an electronic device is provided, comprising: a first memory configured to store a plurality of artificial intelligence models; a second memory configured to load at least one of the plurality of artificial intelligence models stored in the first memory; and a processor configured to: identify an available memory size of the second memory based on receiving a control signal for loading a first artificial intelligence model among the plurality of stored artificial intelligence models into the second memory; and based on the size of the first artificial intelligence model being greater than the available memory size of the second memory, control to obtain a first compressed artificial intelligence model by compressing the first artificial intelligence model based on the available memory size of the second memory, and load the first compressed artificial intelligence model into the second memory.

[0020] The processor can be further configured to: identify whether the performance of the first compressed artificial intelligence model meets predetermined conditions; load the first compressed artificial intelligence model into a second memory based on the performance of the first compressed artificial intelligence model meeting the predetermined conditions; and control to obtain a second compressed artificial intelligence model by compressing the first artificial intelligence model using a method different from the first compressed artificial intelligence model, and load the second compressed artificial intelligence model into the second memory based on the performance of the first compressed artificial intelligence model not meeting the predetermined conditions.

[0021] The performance of the first compressed artificial intelligence model may be any one of, or any combination of, a processor share rate of the first compressed artificial intelligence model, a size of the first compressed artificial intelligence model, and an accuracy of the first compressed artificial intelligence model.

[0022] The electronic device may further include a display, wherein the processor may be further configured to, based on the fact that a plurality of compressed artificial intelligence models obtained by compressing the first artificial intelligence model using a plurality of methods do not satisfy the predetermined condition, control the display to display a message notifying that the predetermined condition is not satisfied.

[0023] The processor can be further configured to, based on the available memory size of the second memory being changed, control to obtain a third compressed artificial intelligence model by compressing the first artificial intelligence model based on the changed available memory size, and load the third compressed artificial intelligence model.

[0024] The processor can be further configured to, based on receiving a control signal for loading the second artificial intelligence model into the second memory, control to load a compressed artificial intelligence model for the first artificial intelligence model obtained by compressing the first artificial intelligence model, and a compressed artificial intelligence model for the second artificial intelligence model obtained by compressing the second artificial intelligence model according to the available memory size of the second memory.

[0025] The processor can be further configured to identify the number of times the multiple artificial intelligence models loaded in the second memory are used based on the multiple artificial intelligence models loaded in the second memory, and load a compressed artificial intelligence model for at least one of the multiple artificial intelligence models based on the identified number of times.

[0026] The electronic device may further include a display, wherein the processor may be further configured to control the display to display a user interface (UI) for receiving the performance of at least one of the plurality of artificial intelligence models, and wherein the first compressed artificial intelligence model may be an artificial intelligence model compressed based on the performance input through the UI.

[0027] The processor may be further configured to control the first memory to store a plurality of compressed artificial intelligence models for the first artificial intelligence model in the first memory.

[0028] The first artificial intelligence model can be compressed using any one of pruning, quantization, decomposition, and knowledge distillation, or any combination thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 is an exemplary diagram illustrating a method of an electronic device for managing a plurality of artificial intelligence models according to an embodiment;

[0030] Figure 2 is a schematic block diagram showing a configuration of an electronic device according to an embodiment;

[0031] Figure 3 is a detailed block diagram showing a configuration of an electronic device according to an embodiment;

[0032] Figure 4A , Figure 4B , Figure 4C and Figure 4D is an example diagram showing a method for loading an artificial intelligence model according to an embodiment;

[0033] Figure 5A , Figure 5B , Figure 5C and Figure 5D is an exemplary diagram illustrating a method for setting conditions for performance of multiple artificial intelligence models according to an embodiment;

[0034] Figure 6 is an example diagram illustrating a process of compressing an artificial intelligence model according to an embodiment;

[0035] Figure 7 is a flow chart showing a compression method according to an embodiment;

[0036] Figure 8 is an example diagram illustrating a method for using an artificial intelligence model according to one or more embodiments;

[0037] Fig. 9A , Fig. 9B and Fig. 9C is an example diagram illustrating a method for using an artificial intelligence model according to an embodiment;

[0038] Fig.10 is a flow chart showing a method for controlling another electronic device according to an embodiment;

[0039] Fig.11 is an exemplary diagram showing operations of an electronic device and an external server according to an embodiment; and

[0040] Fig.12 is a flowchart illustrating an operation of an electronic device downloading a compressed artificial intelligence model from an external server according to an embodiment. DETAILED DESCRIPTION

[0041] Terms used in the specification will be briefly described, and the present disclosure will be described in detail.

[0042] All terms used in this specification, including technical terms and scientific terms, have the same meanings as those commonly understood by those skilled in the relevant art. However, these terms may vary depending on the intentions, legal or technical interpretations and the emergence of new technologies of those skilled in the art. In addition, some terms are arbitrarily selected by the applicant. These terms may be interpreted with the meaning defined here, and unless otherwise specified, may be interpreted based on the full content of this specification and the known technology in the art.

[0043] The present disclosure is not limited to the embodiments disclosed below and can be implemented in various forms, and the scope of the present disclosure is not limited to the following embodiments. In addition, all changes or modifications derived from the meaning and scope of the claims and their equivalents should be interpreted as being included within the scope of the present disclosure. In the following description, configurations that are known but irrelevant to the gist of the present disclosure may be omitted.

[0044] Terms such as "first", "second", etc. may be used to describe various elements, but these elements should not be limited by these terms. These terms are only used to distinguish one element from other elements.

[0045] Singular expressions also include plural meanings as long as they do not have different meanings in the context. In this specification, terms such as "include" and "have / have" should be interpreted as indicating the presence of such features, quantities, operations, elements, components or combinations thereof in the specification, without excluding the presence or possibility of adding one or more other features, quantities, operations, elements, components or combinations thereof.

[0046] In an example embodiment, a "module", "unit" or "part" performs at least one function or operation and may be implemented as hardware such as a processor or an integrated circuit, software executed by a processor, or a combination thereof. In addition, multiple "modules", multiple "units" or multiple "parts" may be integrated into at least one module or chip and may be implemented as at least one processor in addition to the "module", "unit" or "part" that should be implemented in specific hardware.

[0047] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily implement the embodiments. However, the present disclosure may be embodied in many different forms and is not limited to the embodiments described herein. In order to clearly illustrate the present disclosure in the accompanying drawings, some elements that are not necessary for a complete understanding of the present disclosure are omitted for clarity, and throughout the specification, the same reference numerals refer to the same elements.

[0048] An aspect of the embodiments relates to a method for efficiently using multiple artificial intelligence models in a limited memory capacity.

[0049] According to an embodiment, the electronic device compresses the artificial intelligence model in consideration of performance of a plurality of artificial intelligence models.

[0050] Figure 1 is an exemplary diagram illustrating a method of an electronic device for managing multiple artificial intelligence models according to an embodiment.

[0051] For example, according to Figure 1 In the illustrated embodiment, the electronic device 100 uses a facial recognition model, a speech recognition model, and an image recognition model. However, the resources of the electronic device 100, such as memory capacity, GPU / CPU model, are limited, and the size of the artificial intelligence model may be large. Therefore, it will be difficult to use multiple artificial intelligence models as they are. Therefore, the electronic device 100 can compress the artificial intelligence model to reduce the size of the artificial intelligence model. However, when the artificial intelligence model is compressed, the performance of the artificial intelligence model may decrease. In general, the degree of compression of the artificial intelligence model and the performance of the artificial intelligence model may have a trade-off relationship. Therefore, the electronic device 100 can compress a specific artificial intelligence model based on performance and compress another artificial intelligence model based on size. For example, the electronic device 100 can put value on the performance of the frequently used artificial intelligence model, and may not compress or compress it less. However, despite the performance degradation, the electronic device 100 can compress most of the artificial intelligence model that is not often used to reduce its size. Therefore, the electronic device 100 can determine how to compress the artificial intelligence model based on the memory state and the availability of the artificial intelligence model.

[0052] Figure 2 is a schematic block diagram showing a configuration of an electronic device according to an embodiment.

[0053] Reference Figure 2 , the electronic device 100 may include a first memory 110 , a second memory 120 , and a processor 130 .

[0054] For example, the first memory 110 or the second memory 120 may store commands or data related to at least one other constituent element of the electronic device 100. According to one embodiment, the first memory 110 or the second memory 120 may store software and / or programs. The program may include, for example, a kernel, middleware, an application programming interface (API) and / or an application program (or "application"), etc. At least part of the kernel, middleware, and application programming interface (API) may be referred to as an operating system. The kernel may, for example, control or manage system resources for running operations or functions embodied in other programs, and the middleware may, for example, enable the API or application program to communicate with the kernel and send and receive data.

[0055] The first memory 110 may be configured to store a plurality of artificial intelligence models. When the electronic device 100 uses one of the plurality of stored artificial intelligence models, the first memory 110 may send the artificial intelligence model stored in the first memory to the second memory 120. The first memory 110 may be composed of an auxiliary memory device or a second memory device. The first memory 110 may be a non-volatile memory such as a hard disk drive (HDD), a solid state drive (SSD), a universal serial bus (USB), or an optical disk.

[0056] The second memory 120 may receive and load the artificial intelligence model from the first memory 110. The second memory 120 may be composed of a main memory device and a first memory device. The first memory 110 may be composed of a RAM or a ROM.

[0057] The second memory 120 may have a smaller memory size than the first memory 110, and the memory size of the artificial intelligence model may be larger. Therefore, it may be difficult for the second memory 120 to store the artificial intelligence model stored in the first memory 110. Therefore, as described below, the processor 130 may compress the artificial intelligence model and load it to the second memory 120.

[0058] The processor 130 may, for example, control a plurality of hardware components of the software components connected to the processor 130 and perform various data processing and calculations. For example, the processor 130 may be embodied as a system on a chip (SoC). According to an embodiment, the processor 130 may further include a graphics processing unit (GPU) and / or an image signal processor. The processor 130 may load commands or data received from the first memory 110 to the second memory 120 for processing, and store the result data in the first memory 110.

[0059] The processor 130 may include only a processor for artificial intelligence (AI), or may be manufactured as a part of an existing general-purpose processor (e.g., a CPU or an application processor), or may include a graphics processor (e.g., a GPU). The processor used only for artificial intelligence (AI) may be a processor dedicated to probabilistic operations, which has higher parallel processing performance than a general-purpose processor of the related art, thereby quickly performing arithmetic operations in the field of artificial intelligence such as machine training.

[0060] The processor 130 may control the first memory 110 to store the plurality of artificial intelligence models. In a state where the plurality of artificial intelligence models are stored in the first memory 110, a control signal for loading the first artificial intelligence model is received among the plurality of artificial intelligence models, and the processor 130 may identify the available memory size of the second memory 120. The control signal for loading the first artificial intelligence model may be generated in various situations, such as when a user command for using the first artificial intelligence model occurs, a predetermined state for using the first artificial intelligence model, and the like. For example, when a user command for running an application using the first artificial intelligence model is input, a control signal for loading the first artificial intelligence model may be generated. When the first artificial intelligence model needs to be used in a specific situation (e.g., voice input, message reception, receiving a call from a user, image input, a specific time, etc.), a control signal for loading the first artificial intelligence model may be generated.

[0061] When the size of the first artificial intelligence model is larger than the size of the available memory size of the second memory 120, the processor 130 may load a first compressed artificial intelligence model that compresses the first artificial intelligence model based on the available memory size of the second memory 120. The first compressed artificial intelligence model may be compressed by the processor 130, but is not limited thereto. For example, the first compressed artificial intelligence model may be one of the compressed artificial intelligence models pre-stored in the first memory 110.

[0062] The processor 130 may identify whether the performance of the first compressed artificial intelligence model satisfies a predetermined condition. The performance of the first compressed artificial intelligence model may be one of the processor share (CPU / GPU share, CPU / GPU share) of the first compressed artificial intelligence model, the size of the first compressed artificial intelligence model, and the accuracy of the first compressed artificial intelligence model. The predetermined condition may be a condition generated based on the performance of the first compressed artificial intelligence model. However, the performance of the artificial intelligence model is not limited to the above examples. Depending on the type of artificial intelligence model, there may be performance based on various indicators. For example, when the artificial intelligence model is an artificial intelligence model for sentence translation, the performance of the artificial intelligence model may be a performance calculated by a bilingual evaluation understudy (BLEU) score.

[0063] The predetermined condition for the first compressed artificial intelligence model may be one of the following conditions: the share rate of the processor 130 is 20%, the first compressed artificial intelligence model is 128 megabytes (MB) or less, or the accuracy is 80% or more. These are only examples, and the embodiments are not limited to these examples.

[0064] The processor 130 share rate of the artificial intelligence model may indicate the ratio of the amount of calculation used to calculate the artificial intelligence model to the total amount of calculation of the processor 130. The processor 130 share rate may indicate the relative ratio of the artificial intelligence model. According to one embodiment, the above relative share rate is a basis, but is not limited thereto. The electronic device 100 may identify the predetermined condition based on the absolute amount of calculation required to calculate the artificial intelligence model rather than the processor 130 share rate.

[0065] If the performance of the first compressed artificial intelligence model satisfies the predetermined condition, the processor 130 may load the first compressed artificial intelligence model into the second memory 120. If the performance of the first compressed artificial intelligence model does not satisfy the predetermined condition, the processor 130 may load a second compressed artificial intelligence model obtained by compressing the first artificial intelligence model in a different manner into the second memory 120. The processor 130 may load the second compressed artificial intelligence model satisfying the predetermined condition compressed in a manner different from the first compressed artificial intelligence model into the second memory 120.

[0066] If the second compressed artificial intelligence model does not satisfy the predetermined condition, the processor 130 may identify whether the compressed artificial intelligence model obtained by compressing the first artificial intelligence model in another different manner satisfies the predetermined condition. If there is a compressed artificial intelligence model that satisfies the predetermined condition, the processor 130 may load the compressed artificial intelligence model that satisfies the predetermined condition into the second memory 120.

[0067] However, if the multiple compressed artificial intelligence models obtained by compressing the first artificial intelligence model in multiple ways do not satisfy the predetermined conditions, the processor 130 may output a message notifying that the artificial intelligence model does not satisfy the predetermined conditions. For example, if the first artificial intelligence model is compressed a predetermined number of times (for example, 10 times), but there is no compressed artificial intelligence model that satisfies the predetermined conditions, the processor 130 may output a message notifying that there is no compressed artificial intelligence model that satisfies the predetermined conditions. The processor 130 may not load the compressed artificial intelligence model for the first artificial intelligence model into the second memory 120. The processor 130 may load the compressed artificial intelligence model that is most similar to the predetermined condition among the multiple compressed artificial intelligence models generated for the first artificial intelligence model to the second memory 120, and output a message notifying that there is no compressed artificial intelligence model that satisfies the predetermined conditions.

[0068] When the available memory size of the second memory 120 changes, the processor 130 may load a third artificial intelligence model obtained by compressing the first artificial intelligence model based on the changed memory size of the second memory 120 to the second memory 120. For example, when the size of the available memory of the second memory 120 increases, the processor 130 may load a third compressed artificial intelligence model generated based on the increased size of the memory to improve the accuracy of the first artificial intelligence model to the second memory 120. The third compressed artificial intelligence model may have better performance than the first compressed artificial intelligence model or the second compressed artificial intelligence model.

[0069] When receiving a control signal for loading the second artificial intelligence model into the second memory 120, the processor 130 can load the first artificial intelligence model, the compressed artificial intelligence model for the first artificial intelligence model obtained by compressing the second artificial intelligence model, and the compressed artificial intelligence model for the second artificial intelligence model based on the available memory size of the second memory 120.

[0070] For example, when a control signal for loading a second artificial intelligence model is received in a state where the first compressed artificial intelligence model (or the second compressed artificial intelligence model) is loaded into the second memory 120, the processor 130 may load the compressed artificial intelligence model of the second artificial intelligence model into the second memory 120 based on the size of the available memory.

[0071] For example, if the size of the available memory of the second memory 120 is insufficient for compressing and loading the second artificial intelligence model in a state where the first compressed artificial intelligence model (or the second compressed artificial intelligence model) is loaded into the second memory 120, the processor 130 may load the first artificial intelligence model, the compressed artificial intelligence model for the first artificial intelligence model obtained by compressing the second artificial intelligence model, and the compressed artificial intelligence model for the second artificial intelligence model. The processor 130 may load another compressed artificial intelligence model (the first compressed artificial intelligence model or the second compressed artificial intelligence model) that is not the first artificial intelligence model preloaded in the second memory 120 into the second memory 120.

[0072] When the plurality of artificial intelligence models are loaded into the second memory 120, the processor 130 may identify the number of times the plurality of artificial intelligence models loaded into the second memory 120 are used, and load a compressed artificial intelligence model relative to at least one of the plurality of artificial intelligence models based on the identified number. That is, among the plurality of artificial intelligence models loaded into the second memory 120, the processor 130 may increase the compression rate of the artificial intelligence model whose number of uses is less than a predetermined number, and decrease the compression rate of the artificial intelligence model whose number of uses is greater than a predetermined number.

[0073] When the first compressed artificial intelligence model is loaded in the second memory 120, the processor 130 may store the loaded first compressed artificial intelligence model in the first memory 110. The processor 130 may store the loaded compressed artificial intelligence model in the first memory. The processor 130 may store the loaded compressed artificial intelligence model in the first memory to generate a compressed artificial intelligence model later, or receive the compressed artificial intelligence model from the first memory instead of an external source.

[0074] When the electronic device 100 includes the display 150, the processor 130 may control the display 150 to display a user interface (UI) for receiving the performance of at least one artificial intelligence model among the plurality of artificial intelligence models stored in the first memory 110. The first compressed artificial intelligence model may be an artificial intelligence model compressed based on a condition input through the UI for receiving the performance of the first artificial intelligence model.

[0075] The first artificial intelligence model may be compressed using at least one of pruning, quantization, decomposition, and knowledge distillation. Pruning may be a compression method for deleting meaningless weight parameters or weight parameters that do not significantly affect the output value among the weight parameters of the artificial intelligence model. Quantization is a compression technique that quantizes each weight parameter to a predetermined bit. Decomposition may be a compression method for reducing the size of weight parameters by performing an approximate decomposition of a weight matrix or tensor as a set of weight parameters. Knowledge distillation may be a compression method for generating and training a student model that is smaller than the original model by setting the original model as a teacher model. The student model may be generated by the above-mentioned pruning, decomposition, or quantization.

[0076] Figure 3 is a detailed block diagram showing a configuration of an electronic device according to an embodiment.

[0077] The electronic device 100 may further include a first memory 110, a second memory 120, a processor 130, a communicator 140, a display 150, an inputter 160, an audio processor 170, and an audio outputter 180. However, the present disclosure is not limited to the above-mentioned constituent elements, but some constituent elements may be further added or omitted to the extent necessary.

[0078] The first memory 110 and the second memory 120 can store or load an artificial intelligence model. The first memory 110 can be configured by an auxiliary memory device or a second memory device, and the second memory 120 can be configured by a main memory device or a first memory device. The second memory 120 can be composed of a RAM 131 or a ROM 132 in the processor 130.

[0079] The communicator 140 may be configured to perform communication with an external device. For example, the electronic device 100 may receive a plurality of artificial intelligence models from an external server through the communicator 140 and store the plurality of artificial intelligence models in the first memory 110. The electronic device 100 may receive a plurality of compressed artificial intelligence models for corresponding artificial intelligence models through the communicator 140.

[0080] The communicator 140 can communicate with an external device through a third device (e.g., a relay device, a hub, an access point, a server, or a gateway). Wireless communication may include, for example, LTE, LTE Advanced (LTE-A), Code Division Multiple Access (CDMA), Wideband CDMA (WCDMA), Universal Mobile Telecommunications System (UMTS), Wireless Broadband (WiBro), or Global System for Mobile Communications (GMS). According to one embodiment, wireless communication may include, for example, Wireless Fidelity (WiFi), Bluetooth, Bluetooth Low Energy (BLE), ZigBee, Near Field Communication, Magnetic Security Transmission, Frequency (RF), or Body Area Network (BAN). Wired communication may include, for example, at least one of Universal Serial Bus (USB), High Definition Multimedia Interface (HDMI), Recommended Standard 232 (RS-232), Power Line Communication, or Plain Old Telephone Service (POTS). A network that performs wireless or wired communication may include at least one of a telecommunications network (e.g., a computer network (e.g., LAN or WAN), the Internet, or a telephone network).

[0081] The display 150 may be configured to output various images. Specifically, if the plurality of compressed artificial intelligence models obtained by compressing the first artificial intelligence model using a plurality of methods do not satisfy the predetermined condition, the display 150 may display a message notifying that the predetermined condition is not satisfied.

[0082] The display 150 may display a UI for receiving performance of at least one artificial intelligence model among a plurality of artificial intelligence models stored in the first memory 110 .

[0083] The display 150 for providing various images can be implemented with various types of display panels. For example, the display panel can be a liquid crystal display (LCD), an organic light emitting diode (OLED), an active matrix organic light emitting diode (AM-OLED), a liquid crystal on silicon (LCOS), or a digital light processing (DLP). The display 150 can also be coupled to at least one of the front area, the side area, and the rear area of ​​the electronic device 100 in the form of a flexible display.

[0084] The inputter 160 may be configured to receive a user command. The inputter 160 may include a camera 161 , a microphone 162 , and a touch panel 163 .

[0085] The camera 161 may be configured to obtain image data near the electronic device 100. The camera 161 may capture still images or videos. For example, the camera 161 may include at least one image sensor (e.g., a front sensor or a rear sensor), a lens, an image signal processor (ISP), or a flash (e.g., an LED, a xenon lamp, etc.). The microphone 162 may be configured to obtain noise near the electronic device 100. The microphone 162 may receive an external source signal and generate voice information. The microphone 162 may use various noise removal algorithms for removing noise generated when receiving an external sound signal. The image information or voice information input by the camera 161 and the microphone 162 may be input as an input value of the artificial intelligence model.

[0086] The touch panel 163 may be configured to receive various usage inputs. The touch panel 163 may receive data through user operations. The touch panel 163 may be combined with a display to be described below.

[0087] The inputter 160 may be variously configured to receive various data in addition to the camera 161 , the microphone 162 , and the touch panel 163 .

[0088] The audio processor 170 may be configured to process audio data. The audio processor 170 may perform various processes such as decoding, amplification, and noise filtering of audio data. The audio data processed by the audio processor 150 may be output to the audio outputter 180.

[0089] The audio output 180 may be configured to output various alarm voices and voice messages in addition to various audio data after various processes such as decoding, amplification, and noise filtering are performed by the audio processor 170. The audio output 180 may be implemented as a speaker, but is not limited thereto. The audio output 180 may be implemented as an output terminal for outputting audio data.

[0090] The processor 130 may control the overall operation of the electronic device 100. The processor 130 may include a RAM 131, a ROM 132, a CPU 133, a graphics processing unit (GPU) 134, first to nth interfaces 135-1 to 135-n, and a bus 136. The RAM 131, the ROM 132, the CPU 133, the graphics processing unit (GPU) 134, the first to nth interfaces 135-1 to 135-n, etc. may be connected to each other via the bus 136.

[0091] A set of instructions for system booting and the like may be stored in the ROM 132. When a power-on command is input and power is supplied, the CPU 133 may copy an O / S stored in a memory to the RAM 131, run the O / S, and perform system booting according to one or more instructions stored in the ROM 132. When the system booting is completed, the CPU 133 may copy various programs stored in the memory to the RAM 131, run the programs copied to the RAM 131, and perform various operations.

[0092] The CPU 133 may access the first memory 110 or the second memory 120 and perform booting by using an operating system (O / S) stored in the first memory 110 or the second memory 120. The CPU 133 may perform various operations by using various programs, contents, data, etc. stored in the first memory 110 or the second memory 120.

[0093] The first to nth interfaces 135-1 to 135-n may be connected to various constituent elements. One of the interfaces may be a network interface connected to an external device through a network.

[0094] In the following, reference will be made to FIG. 4A to FIG. 10 One or more example embodiments are described.

[0095] Figure 4A , Figure 4B , Figure 4C and Figure 4D is an example diagram showing a method for loading an artificial intelligence model according to one embodiment.

[0096] Reference Figure 4A , the available memory size of the second memory 120 may be 30MB. If a control signal for loading the image recognition model 410 is received, the electronic device 100 may recognize the size of the available memory of the second memory 120. The size of the image recognition model 410 may be 50MB, and the size of the available memory of the second memory 120 may be 30MB. Therefore, the electronic device 100 may compress the image recognition model 410 and load the image recognition model to the second memory 120. Figure 4A On the right side, the size of the compressed image recognition model 420 may be 30MB. In this case, the accuracy of the compressed image recognition model 420 may be reduced, but the image recognition model may be loaded according to the size of the available memory of the second memory 120.

[0097] Reference Figure 4B , the electronic device 100 may output a message notifying that the image recognition performance is degraded because the memory size is small.

[0098] When the size of the available memory of the second memory 120 changes, the electronic device 100 may compress the image recognition model again. Figure 4C , when the face recognition model, the speech recognition model, and the compressed image recognition model 420 are loaded in the second memory 120 and the face recognition model is deleted, the size of the available memory of the second memory 120 can be increased to 100MB. Therefore, the electronic device 100 can load the new compressed image recognition model 430 to the second memory 120 instead of the compressed image recognition model 420 with degraded performance. Figure 4C The compressed image recognition model 430 is shown to have Figure 4A The image recognition model 410 of the present invention has the same performance as that of the image recognition model 410, but is not limited thereto. For example, to the extent necessary, the electronic device 100 may load an image recognition model having a share rate of 7%, a model size of 100MB, and an accuracy rate of 85% to the second memory 120. In this case, performing compression to the extent necessary may be related to a predetermined condition for the performance of the artificial intelligence model as described below.

[0099] Reference Figure 4C , the electronic device 100 can output Figure 4D The message shown is used to notify that the image recognition performance has improved again.

[0100] Figure 5A , Figure 5B , Figure 5C and Figure 5D is an example diagram illustrating a method for setting conditions for performance of multiple artificial intelligence models according to one embodiment.

[0101] Reference Figure 5A , the electronic device 100 may display a UI screen for changing the performance and performance conditions of multiple artificial intelligence models stored in the first memory 110. The facial recognition model may have a performance of a share rate of 32.3%, a model size of 100MB, and an accuracy rate of 87%. The speech recognition model may have a performance of a share rate of 10.3%, a model size of 80MB, and an accuracy rate of 83% of the processor 130. The image recognition model may have a performance of a share rate of 5.3%, a model size of 50MB, and an accuracy rate of 72% of the processor 130.

[0102] If a user command is received through the change element 510 for changing the performance condition of the face recognition model, the electronic device 100 may display Figure 5A. The UI screen for changing the performance conditions of the facial recognition model shown on the right side of . The UI screen for changing the performance conditions may include information about a UI bar for controlling performance, the accuracy of the most recent facial recognition, the size of the facial recognition model, and the amount of calculation. The performance of the facial recognition model may be improved by moving the UI bar to the right, and the performance of the facial recognition model may be reduced by moving the UI bar to the left. The artificial intelligence model performance corresponding to the position of the UI bar may be a predetermined performance condition. The share rate of the processor 130, the size and accuracy of the artificial intelligence model may be pre-stored according to the position of each UI bar. Therefore, the user can simply control the performance of the artificial intelligence model by controlling the UI bar.

[0103] Reference Figure 5B , the UI screen for changing performance conditions may further display the results of the most recent facial recognition. Figure 5B , the electronic device 100 can display the facial recognition result of the specific image and guide the user to control the performance of the user recognition model as shown by UI elements 521 and 522.

[0104] The electronic device 100 may display a screen for recommending a compressed artificial intelligence model according to the current state of the second memory 120 instead of displaying a UI bar for controlling the performance of the facial recognition model. Figure 5C , the electronic device 100 may display a UI screen for recommending an artificial intelligence model compressed based on the available memory size of the second memory 120. If a UI element 530 for selecting one of the recommended multiple artificial intelligence models is selected, the electronic device 100 may display a screen such as Figure 5D The UI screen is shown on the right.

[0105] Reference FIG. 5A to FIG. 5C , the electronic device 100 can recommend an artificial intelligence model compressed based on the state of the second memory 120 or the share state of the processor 130 of the artificial intelligence model, and can store the size and accuracy of the artificial intelligence model to identify the performance condition of the artificial intelligence model.

[0106] Therefore, the electronic device 100 may display a UI screen for controlling each performance condition (the share rate of the processor 130 of the artificial intelligence model, the size and accuracy of the artificial intelligence model).

[0107] Reference Figure 5D On the left side of the electronic device 100, if a user command for selecting a UI element 541 for controlling the performance of a facial recognition model among a plurality of artificial intelligence models loaded in the second memory 120 is input, the electronic device 100 may display a Figure 5D, and a UI for controlling each performance condition is shown on the right side of . The performance conditions of the artificial intelligence model may be a maximum share rate of the processor 130 of the compressed artificial intelligence model, a maximum size of the compressed artificial intelligence model, and a minimum accuracy condition of the compressed artificial intelligence model. If a user command is input through a UI bar 542 for controlling the maximum share rate of the processor 130 of the compressed artificial intelligence model, a UI bar 543 for controlling the maximum size of the compressed artificial intelligence model, and a UI bar 544 for controlling the minimum accuracy of the compressed artificial intelligence model, the electronic device 100 can compress the artificial intelligence model according to the input conditions.

[0108] For example, when the maximum share rate of the processor 130 of the compressed artificial intelligence model is 35%, the maximum size of the compressed artificial intelligence model is 100MB, and the minimum accuracy of the compressed artificial intelligence model is 80%, the artificial intelligence model compressed by the electronic device can be 35% or less, the model size can be 100MB or less, and the accuracy can be 80% or higher.

[0109] If there is a compressed artificial intelligence model that meets the input condition, the electronic device 100 may display a message notifying that the artificial intelligence model is compressed according to the predetermined condition. If there is no compressed artificial intelligence model that meets the input condition, the electronic device 100 may display a message notifying that the compression of the artificial intelligence model fails. For example, referring to Figure 5D If there is a compressed artificial intelligence model that meets the input conditions, the electronic device can output a message "The performance of the facial recognition model is adjusted". If there is no compressed artificial intelligence model that meets the input conditions, the electronic device can output a message "The performance adjustment of the facial recognition model failed, please change the requirements".

[0110] Figure 5A , Figure 5B , Figure 5C and Figure 5D The method for controlling the performance of the artificial intelligence model loaded in the second memory 120 is shown, but is not limited thereto. The electronic device 100 may control the performance of each artificial intelligence model stored in the first memory 110.

[0111] Figure 6 is an example diagram illustrating a process of compressing an artificial intelligence model according to an embodiment.

[0112] Reference Figure 5A , Figure 5B , Figure 5C and Figure 5DIn step S610, the electronic device 100 may control the performance of the plurality of artificial intelligence models. For example, the electronic device 100 may set conditions for the maximum share rate of the processor 130 for the compressed artificial intelligence model, the maximum size of the compressed artificial intelligence model, and the minimum accuracy of the compressed artificial intelligence model for each artificial intelligence model.

[0113] In step S620, when a control signal for compressing at least one of the plurality of artificial intelligence models is received, the electronic device 100 may compress at least one artificial intelligence model. The artificial intelligence model may be compressed by at least one of pruning, quantization, decomposition, and knowledge distillation. The artificial intelligence model may be compressed by the electronic device 100, but may be compressed by an external server so that the electronic device 100 receives the compressed artificial intelligence model.

[0114] In step S630, the electronic device 100 may identify whether the compressed artificial intelligence model satisfies a predetermined condition. If in step S630-Y, the compressed artificial intelligence model satisfies the predetermined condition, the electronic device 100 may load the compressed artificial intelligence model into the second memory. If in step S630-N, the compressed artificial intelligence model does not satisfy the predetermined condition, the electronic device 100 may identify whether the artificial intelligence model has been compressed more than a predetermined number of times in step S650.

[0115] If the artificial intelligence model has not been compressed more than the predetermined number of times in step S650-N, the electronic device 100 may return to step S620 and compress the artificial intelligence model again. The electronic device 100 may compress the artificial intelligence model using a method different from the previous compression method.

[0116] If the artificial intelligence model is compressed more than the predetermined number of times in step S650-Y, the electronic device 100 may output a notification of compression failure of the artificial intelligence model in step S660. In other words, if the predetermined condition is not satisfied even though the artificial intelligence model is compressed more than the predetermined number of times, the electronic device 100 may recognize that it is impossible to compress the artificial intelligence model to satisfy the predetermined condition, and output compression failure of the artificial intelligence model. The electronic device 100 may further output a message requesting a change in the predetermined condition.

[0117] Figure 7 is a flow chart illustrating a compression method according to an embodiment.

[0118] Reference Figure 7 According to one embodiment, in a state where at least one artificial intelligence model is loaded in the second memory 120, another artificial intelligence model (e.g., the first artificial intelligence model) is loaded.

[0119] In step S710, the electronic device 100 may receive a command for loading the first artificial intelligence model. In step S720, the electronic device 100 may identify whether the resource can be used. Identifying whether the resource can be used can be defined by identifying whether the first artificial intelligence model satisfies performance conditions such as the available memory size of the second memory 120, the share rate (CPU or GPU share rate) of the available processor 130, etc.

[0120] If it is identified in step S720-Y that the resource is available, the electronic device 100 may load the first artificial intelligence model to the second memory 120 in step S780. If it is identified in step S720-N that the resource is not available, the electronic device 100 may compress the first artificial intelligence model in step S730. In step S740, the electronic device 100 may identify whether the resource is available with respect to the compressed first artificial intelligence model.

[0121] If it is identified in step S740-Y that the resource is available, the electronic device 100 may load the compressed first artificial intelligence model to the second memory 120 in step S780. If it is identified in step S740-N that the resource is not available, the electronic device 100 may compress another artificial intelligence model loaded in the second memory 120. Figure 6 , if the resource is not available, the electronic device 100 may compress the first artificial intelligence model in a different way. Figure 7 , the electronic device 100 may compress another artificial intelligence model loaded in the second memory 120 in addition to the first artificial intelligence model, instead of compressing the first artificial intelligence model in a different method for loading the artificial intelligence model into the second memory 120.

[0122] If another artificial intelligence model is compressed, the electronic device 100 may identify whether resources are available at step S760. The electronic device 100 may identify whether resources increased by another compressed artificial intelligence model are suitable for loading the first artificial intelligence model compressed at step S730.

[0123] When it is identified that the resource is available in step S760-Y, the electronic device 100 may load the other artificial intelligence model to the second memory 120 in step S780. If it is identified that the resource is not available, the electronic device 100 may output a compression failure of the artificial intelligence model in step S770.

[0124] exist Figure 6 and Figure 7 The examples described in can be performed independently of each other, but are not limited to this. For example, in the execution of Figure 6 After step S650 is executed Figure 7If it is determined in step S740-N that the resource is not available for the first artificial intelligence model compressed according to step S730, the electronic device 100 may compress the first artificial intelligence model up to a predetermined number of times in a different method, as in step S650. That is, if it is determined that the resource is still unavailable even if the first artificial intelligence model is compressed more than a predetermined number of times, the electronic device 100 may compress another artificial intelligence model loaded in the second memory 120 according to step S750.

[0125] Figure 8 is an example diagram illustrating a method for using an artificial intelligence model according to one or more embodiments.

[0126] Reference Figure 8 , the electronic device 100 can exchange pictures and conversations with each other by running a messenger application. The image analysis model 810, the conversation recognition model 820, and the keyboard recommendation model 830 can be loaded into the second memory 120 of the electronic device 100. The conversation recognition model may be a model for continuously analyzing the conversation content to understand the conversation content between each other if a conversation is input through the messenger application. The conversation recognition model may manage language understanding (NLU) and context. The image analysis model may be a model for analyzing an image received through the messenger application. The image analysis model may analyze objects included in the image or classify the analyzed objects. The keyboard recommendation model may be a model for recommending words or sentences based on a conversation scenario identified by the conversation recognition model. The keyboard recommendation model may predict and recommend natural language understanding (NLU) and key input of the user.

[0127] For example, the electronic device 100 may analyze the conversation and recommend an action to be performed by the user. The electronic device 100 may input conversations such as "Photos taken yesterday!", "Wow, did YY go there too?", and "Do you have any other photos?" into the conversation recognition model as input values, and recommend actions such as "Send the photos taken yesterday with YY to others". In this case, the electronic device 100 may relatively improve the performance of the conversation management model by compressing the image analysis model and the keyboard recommendation model to low performance.

[0128] The electronic device 100 can analyze the image and the conversation and perform the action to be performed by the user. The electronic device 100 can input conversations such as "Photos taken yesterday!", "Wow, did YY go there too?" and "Do you have other photos?" into the conversation recognition model as input values, and recommend actions such as "Send the photos taken yesterday with YY to others." The electronic device 100 can identify YY included in the image through image analysis, and additionally obtain an image including YY in the photos taken yesterday to provide. The electronic device 100 can compress the keyboard recommendation model to low performance, and relatively improve the performance of the conversation management model and the image analysis model.

[0129] The electronic device 100 can recommend words or sentences to the user by analyzing images and conversations. The electronic device 100 can input conversations such as "Photos taken yesterday!", "Wow, did YY go there too?" and "Do you have any other photos?" into the conversation recognition model as input values, and recommend actions such as "Send the photos taken yesterday with YY to others".

[0130] The electronic device 100 can identify YY included in the image through image analysis, and additionally obtain an image including YY in the photo taken yesterday to provide. When there are photos taken yesterday with YY, the electronic device 100 can recommend the words "Yes, I have!". In this case, the image analysis model, the dialogue management model, and the keyboard recommendation model can be compressed to have a certain level of performance.

[0131] Reference Figure 8 In an example embodiment, the performance of each model can be referenced by the user FIG. 5A to FIG. 5D For example, if the user does not use the keyboard recommendation model, the electronic device 100 may compress each artificial intelligence model according to the user setting. The electronic device 100 may not load the keyboard recommendation model to the second memory 120, but only load the dialogue recognition model and the image analysis model.

[0132] Even if the user does not set the performance of the artificial intelligence model, the electronic device 100 can identify the performance of the artificial intelligence model based on a specific scenario, history, context, etc. For example, the electronic device 100 can identify the performance and compression rate of the artificial intelligence model based on the number of times each artificial intelligence model is used. For example, when the user does not use the result value output by the keyboard recommendation model, the electronic device 100 can identify the performance of the keyboard recommendation model as low (e.g., from 50% accuracy to minimum memory capacity).

[0133] The electronic device 100 may be a display device. When the electronic device 100 uses super-resolution technology or screen analysis technology for a product ordering service, an artificial intelligence model may be necessary. In general, the size of the second memory 120 of the display device may not be large, so it may be necessary to control the performance of the artificial intelligence model according to the situation. For example, when a screen analysis model is used for a product ordering service, the electronic device 100 may use the screen analysis model only in specific circumstances (for example, when a TV program is played again in the afternoon). If the user does not use the product ordering service, the electronic device 100 may compress the product ordering service model to have lower performance and load the product ordering service model into the second memory 120.

[0134] The electronic device 100 may be an artificial intelligence speaker or an artificial intelligence robot. The electronic device 100 may require limited word or object recognition used only in a specific place (e.g., a house). Therefore, the electronic device 100 may limit the category of recognized words or models related to the object recognition category to control performance.

[0135] Fig. 9A , Fig. 9B and Fig. 9C is an example diagram illustrating a method for using an artificial intelligence model according to an embodiment.

[0136] As described above, the electronic device 100 can compress the artificial intelligence model not only based on the performance conditions set by the user, but also based on specific situations, histories, contexts, etc. For example, when multiple artificial intelligence models used by the electronic device 100 are used independently or in a specific order, the electronic device 100 can identify the compression order of the multiple artificial intelligence models according to the specific order.

[0137] For example, consider Fig. 9A , Fig. 9B and Fig. 9C The Internet of Things (IoT) system shown. Fig. 9A , Fig. 9B and Fig. 9C The IoT system shown relates to an artificial intelligence system that recognizes and notifies a specific situation in response to a user command to notify a specific situation. Specifically, according to the IoT system, if it receives a user's natural language command "Please let me know when the water boils" and senses that the water is boiling through an IoT camera in the kitchen, the user can be notified of this situation. The user's natural language command "Please tell me when the water boils" can be recognized by an automatic speech recognition model (ASR model), and sensing that the water in the pot is boiling can be recognized by an object detection model. The reply to the user can be generated by a language generation model.

[0138] For convenience, it will be assumed that the second memory 120 is 128 MB, and each artificial intelligence model is as follows. The embodiment is not limited thereto.

[0139] [Table 1]

[0140]

[0141] "Weight parameter #" may indicate the number of weight parameters of the artificial intelligence parameter, and data representation bits may indicate the number of bits required to store one weight parameter. The model size may be obtained by multiplying the weight parameter # by the data representation bits. To reduce the number of weight parameters, a pruning method may be used, and a quantization method may be used to reduce the bits required to store one weight parameter.

[0142] Reference Fig. 9A , the electronic device 100 may receive a user command such as "Please tell me when the water boils." In this case, the object detection model and the language generation model may not be suitable for analyzing the user command. Fig. 9A , the electronic device 100 can maintain the size of the ASR model and compress other artificial intelligence models. For example, the electronic device 100 can compress each artificial intelligence model as shown in Table 2 below and load the artificial intelligence model into the second memory 120.

[0143] [Table 2]

[0144]

[0145] The second memory 120 may have a size of 128MB, and the size of the ASR model may be 80MB. Therefore, the electronic device 100 may not compress the ASR model but load the ASR model as is to the second memory 120. The electronic device 100 may compress the object detection model and the language generation model as in Table 2 and load them to the second memory 120.

[0146] When a user command is input, the electronic device 100 can identify whether the water in the pot is boiling according to the user command. Fig. 9B , the electronic device 100 may receive an image of the pot from a camera included in the IoT system and analyze the received image. The electronic device 100 itself may photograph the pot and analyze the image.

[0147] In this case, ASR models and language generation models may not be suitable for image analysis. Fig. 9B, the electronic device 100 can compress other artificial intelligence models while maintaining the size of the object detection model. The model size of the object detection model can be 400MB. Therefore, its size can be larger than the size of the second memory 120. Therefore, the electronic device 100 can compress the object detection model as shown in Table 3 below and load the model to the second memory 120.

[0148] [Table 3]

[0149]

[0150] The second memory 120 may have a size of 128 MB, and the model size of the object detection model may be 400 MB. Therefore, the electronic device 100 may compress the object detection model based on the available memory size of the second memory 120 and load the model to the second memory 120, and compress the ASR model and the language generation model as shown in Table 3 to load the models to the second memory 120.

[0151] If boiling of water in the pot is sensed, the electronic device 100 may output a message notifying the user that the water is boiling. Fig. 9C , the electronic device 100 can output a voice such as “The water in the pot is boiling!”

[0152] In this case, the ASR model and the object detection model are not suitable for generating output speech. Fig. 9C , the electronic device 100 can compress other artificial intelligence models while maintaining the size of the language generation model. For example, the electronic device 100 can compress each artificial intelligence model as shown in Table 4 below and load the artificial intelligence model into the second memory 120.

[0153] [Table 4]

[0154]

[0155] The second memory 120 may have a size of 128MB, and the language generation model may have a size of 120MB, and the electronic device 100 may not compress the language generation model, but may load the model as is to the second memory 120. The electronic device 100 may compress the ASR model and the object detection model as in Table 4, and load the models to the second memory 120.

[0156] According to one embodiment, the electronic device 100 may directly compress multiple artificial intelligence models, or request compression of the artificial intelligence model from an external server. When the electronic device 100 requests compression of the artificial intelligence model from the external server, the electronic device 100 may receive the compressed artificial intelligence model from the external server and load the artificial intelligence model into the second memory 120. When the electronic device 100 directly compresses the artificial intelligence model, different compression methods may be used depending on the situation. For example, when fast compression is requested, the electronic device 100 may compress the artificial intelligence model using a compression method that does not require additional training (e.g., pruning, etc.). If fast compression is not required, the electronic device 100 may compress the artificial intelligence model using various methods in consideration of accuracy.

[0157] According to an embodiment, the electronic device 100 may store various compressed artificial intelligence models in the first memory 110 and load a necessary compressed artificial intelligence model from the first memory 110 to the second memory 120. The form of the compressed artificial intelligence model stored in the first memory 110 may vary.

[0158] For example, when compressing an artificial intelligence model according to a predetermined condition, the electronic device 100 (or an external server) may store the compressed artificial intelligence model in the first memory 110. When a compressed artificial intelligence model having a size similar to that of the compressed artificial intelligence model is pre-stored in the first memory 110, the electronic device 100 may compare the performance of the pre-stored compressed artificial intelligence model with the performance of the newly compressed artificial intelligence model, and store the compressed artificial intelligence model having better performance in the first memory 110.

[0159] When downloading the initial artificial intelligence model from the external server, the electronic device 100 can download various compressed artificial intelligence models related to the downloaded artificial intelligence model. For example, when the artificial intelligence model is 64MB, the electronic device 100 can obtain: a compressed artificial intelligence model of 32MB, which is half of the 64MB artificial intelligence model; a compressed artificial intelligence model of 16MB, which is half of the 32MB compressed artificial intelligence model; a compressed artificial intelligence model of 8MB, which is half of the 16MB compressed artificial intelligence model; a compressed artificial intelligence model of 4MB, which is half of the 8MB artificial intelligence model; a compressed artificial intelligence model of 2MB, which is half of the 4MB artificial intelligence model; and a compressed artificial intelligence model of 1MB, which is half of the 2MB artificial intelligence model. The electronic device 100 can store 6 (six) compressed artificial intelligence models in the first memory 110 relative to the 64MB artificial intelligence model. When storing a compressed artificial intelligence model of half the size, the space in which one artificial intelligence model and the compressed artificial intelligence model are stored can be equal to or less than 2 times the size of the original artificial intelligence model. For example, the sum of the size of a 64MB artificial intelligence model and the size of 6 (six) compressed artificial intelligence models may be 127MB, and the electronic device 100 may store various compressed artificial intelligence models with a small capacity.

[0160] Fig.10 is a flow chart illustrating a method for controlling another electronic device according to an embodiment.

[0161] In step S1010, the electronic device 100 may store a plurality of artificial intelligence models in the first memory 110. As described above, a compressed artificial intelligence model for each of the plurality of artificial intelligence models may be stored together.

[0162] In step S1020, when a control signal for loading a first artificial intelligence model among the plurality of stored artificial intelligence models in the second memory 120 is received, the electronic device 100 may identify an available memory size of the second memory. The control signal for loading the first artificial intelligence model in the second memory 120 may be received when a user command is input. The control signal for loading the first artificial intelligence model in the second memory 120 may be received when a predetermined condition such as a specific situation, history, context, etc. is satisfied.

[0163] In step S1030 , when the size of the first artificial intelligence model is greater than the available memory size of the second memory 120 , the electronic device 100 may load a first compressed artificial intelligence model obtained by compressing the first artificial intelligence model based on the available memory size in the second memory 120 .

[0164] Fig.11is an exemplary diagram illustrating operations of an electronic device and an external server according to an embodiment.

[0165] As described above, the compressed artificial intelligence model may be obtained by the electronic device 100 , but is not limited thereto. The electronic device 100 may download and obtain the compressed artificial intelligence model from the external server 200 .

[0166] The electronic device 100 can directly obtain the artificial intelligence model compressed as needed, or download the artificial intelligence model from the external server 200. For example, when the artificial intelligence model to be compressed has a predetermined size or larger so that the amount of calculation required to compress the artificial intelligence model to be compressed by the electronic device 100 is greater than the predetermined amount of calculation, the electronic device 100 can download the compressed artificial intelligence model from the external server 200. Regardless of the size of the artificial intelligence model to be compressed, if the amount of calculation required to compress the artificial intelligence model is greater than the predetermined amount of calculation, the electronic device 100 can download the artificial intelligence model from the external server 200. In addition, the electronic device 100 can identify whether the electronic device compresses the artificial intelligence model or downloads the artificial intelligence model from the external server 200 based on the processor share rate.

[0167] Fig.12 is a flowchart illustrating an operation of an electronic device downloading a compressed artificial intelligence model from an external server according to an embodiment.

[0168] In step S1210, the electronic device 100 may receive a control signal for compressing the artificial intelligence model. The control signal for compressing the artificial intelligence model may be received under various conditions. For example, when the artificial intelligence model is received from the external server 200, the electronic device 100 may receive a control signal for compressing the downloaded artificial intelligence model. When a control signal for loading the artificial intelligence model to the second memory 120 is received and the artificial intelligence model is compressed according to the control signal for loading the artificial intelligence model to the second memory 120, the electronic device 100 may receive a control signal for compressing the artificial intelligence model.

[0169] The electronic device 100 can identify whether the artificial intelligence model is compressed in the electronic device 100 or the external server 200 when downloading the artificial intelligence model or whether the artificial intelligence model needs to be compressed, and can identify whether the artificial intelligence model is compressed in the electronic device 100 or the external server 200.

[0170] In step S1220, the electronic device 100 may identify whether to compress the artificial intelligence model in the electronic device 100. When it is identified in step S1220-Y that the artificial intelligence model is compressed in the electronic device 100, the electronic device 100 may compress the artificial intelligence model and store the compressed artificial intelligence model in the first memory 110 or load the artificial intelligence model in the second memory 120 in step S1250. When it is identified in step S1220-N that the artificial intelligence model is compressed in the external server 200, the electronic device 100 may request the external server 200 to compress the artificial intelligence model in step S1230. As described above, when the artificial intelligence model to be compressed is a predetermined size or larger and the amount of calculation required to compress the artificial intelligence model to be compressed by the electronic device is equal to or greater than the predetermined amount of calculation, the electronic device 100 may request the external server 200 to compress the artificial intelligence model. When the amount of calculation required to compress the artificial intelligence model is equal to or greater than the predetermined amount of calculation regardless of the size of the artificial intelligence model to be compressed, the electronic device 100 may request the external server 20 to compress the artificial intelligence model. The electronic device 100 may identify whether to compress the artificial intelligence model or request the external server 200 to compress the artificial intelligence model based on the processing share rate.

[0171] In step S1240 , the electronic device 100 may download the compressed artificial intelligence model from the external server 200 and store the artificial intelligence model in the first memory 110 , or load the artificial intelligence model to the second memory 120 .

[0172] Although the present disclosure has been described so that all elements of the embodiment are connected to each other or operate in combination, it will be understood that the present disclosure is not limited to the disclosed embodiments. That is, within the scope of the present disclosure, all elements can be selectively connected to each other in one or more combinations. In addition, although all elements can be implemented as an independent hardware, some or all elements can be selectively combined to be embodied as a computer program, which includes a program module that performs some or all functions in one or more hardware.

[0173] At least part of the devices (e.g., modules or functions) or methods (e.g., operations) according to various embodiments may be embodied as one or more instructions stored in a non-transitory computer-readable medium in the form of a program module. When a processor (e.g., processor 130) runs one or more instructions, the processor may perform functions corresponding to the one or more instructions.

[0174] The program may be stored in a computer-readable recording medium and may be read and executed by a computer to implement one or more embodiments.

[0175] The non-transitory readable recording medium refers to a medium that stores data semi-permanently and can be read by a device, and may also include registers, caches, buffers, etc., but does not include transmission media such as signals, currents, etc.

[0176] Specifically, the above-mentioned program may be stored in a non-transitory readable recording medium such as a CD, a DVD, a hard disk, a Blu-ray disc, a USB, an internal memory (eg, the memory 150 ), a memory card, a ROM, or a RAM.

[0177] Furthermore, the method according to the disclosed embodiments may be provided as a computer program product.

[0178] The computer program product may include an S / W program, a computer-readable storage medium in which the S / W program is stored, or a commodity traded between a buyer and a seller.

[0179] For example, a computer program product may include a product (e.g., a downloadable application) in the form of a S / W program distributed online by a manufacturer and an electronic market (e.g., Google Play Store, App Store, etc.). In the case of online distribution, at least a portion of the S / W program may be stored in a storage medium, or created temporarily. In this case, the storage medium may be a storage medium of a server or relay server of the manufacturer or electronic market.

[0180] Although embodiments have been shown and described, it will be appreciated by those skilled in the art that changes may be made to these embodiments without departing from the principles and spirit of the present disclosure. Therefore, the scope of the present disclosure is not to be interpreted as being limited to the described embodiments, but is to be defined by the appended claims and their equivalents.

Claims

1. A method for controlling an electronic device, the method include: storing a plurality of artificial intelligence models in a first memory; identifying an available memory size of the second memory based on receiving a control signal for loading a first artificial intelligence model among the plurality of stored artificial intelligence models into the second memory; Based on the size of the first artificial intelligence model being smaller than the available memory size of the second memory, loading the first artificial intelligence model into the second memory; Based on the size of the first artificial intelligence model being greater than the available memory size of the second memory, obtaining a first compressed artificial intelligence model by compressing the first artificial intelligence model based on the available memory size of the second memory; identifying whether the performance of the first compression artificial intelligence model meets a predetermined condition; as well as Based on the performance of the first compressed artificial intelligence model satisfying a predetermined condition, the first compressed artificial intelligence model is loaded into the second memory.

2. The method of claim 1, further comprising: include: Based on the fact that the first compressed artificial intelligence model does not satisfy a predetermined condition, a second compressed artificial intelligence model is obtained by compressing the first artificial intelligence model using a method different from the first compressed artificial intelligence model, and the second compressed artificial intelligence model is loaded into a second memory.

3. The method of claim 2, wherein the performance of the first compressed artificial intelligence model is any one of a processor share rate of the first compressed artificial intelligence model, a size of the first compressed artificial intelligence model, and an accuracy of the first compressed artificial intelligence model, or any combination thereof.

4. The method of claim 2, further comprising: include: Based on the fact that the plurality of compressed artificial intelligence models obtained by compressing the first artificial intelligence model using the plurality of methods do not satisfy the predetermined condition, a message notifying that the predetermined condition is not satisfied is displayed.

5. The method of claim 1, further comprising: include: Based on the available memory size of the second memory being changed, a third compressed artificial intelligence model is obtained by compressing the first artificial intelligence model based on the changed available memory size, and the third compressed artificial intelligence model is loaded.

6. The method of claim 1, further comprising: include: Based on receiving a control signal for loading the second artificial intelligence model into the second memory, a compressed artificial intelligence model for the first artificial intelligence model obtained by compressing the first artificial intelligence model and a compressed artificial intelligence model for the second artificial intelligence model obtained by compressing the second artificial intelligence model are loaded according to the available memory size of the second memory.

7. The method of claim 1, further comprising: include: Based on loading the plurality of artificial intelligence models in the second memory, identifying a number of times the plurality of artificial intelligence models loaded in the second memory are used; as well as Based on the identified number of usages, a compressed artificial intelligence model is loaded for at least one of the plurality of artificial intelligence models.

8. The method of claim 1, further comprising: include: displaying a user interface UI for receiving performance of at least one of the plurality of artificial intelligence models, The first compressed artificial intelligence model is an artificial intelligence model compressed based on the performance input through the UI.

9. The method of claim 1, wherein storing further comprises storing a plurality of compressed artificial intelligence models for a first artificial intelligence model in a first memory.

10. The method of claim 1, wherein the first artificial intelligence model is compressed using any one of pruning, quantization, decomposition, and knowledge distillation, or any combination thereof.

11. An electronic device, include: A first memory configured to store a plurality of artificial intelligence models; a second memory configured to load at least one of the plurality of artificial intelligence models stored in the first memory; as well as Processor, configured as: identifying an available memory size of the second memory based on receiving a control signal for loading a first artificial intelligence model among the plurality of stored artificial intelligence models into the second memory; Based on the size of the first artificial intelligence model being smaller than the available memory size of the second memory, loading the first artificial intelligence model into the second memory; Based on the size of the first artificial intelligence model being greater than the available memory size of the second memory, controlling to obtain a first compressed artificial intelligence model by compressing the first artificial intelligence model based on the available memory size of the second memory, identifying whether the performance of the first compression artificial intelligence model satisfies a predetermined condition, and Based on the performance of the first compressed artificial intelligence model satisfying a predetermined condition, the first compressed artificial intelligence model is loaded into the second memory.

12. The electronic device of claim 11, wherein the processor is further configured to: Based on the fact that the performance of the first compressed artificial intelligence model does not meet the predetermined conditions, control is performed to obtain a second compressed artificial intelligence model by compressing the first artificial intelligence model using a method different from the first compressed artificial intelligence model, and load the second compressed artificial intelligence model into the second memory.

13. The electronic device of claim 12, wherein the performance of the first compressed artificial intelligence model is any one of a processor share rate of the first compressed artificial intelligence model, a size of the first compressed artificial intelligence model, and an accuracy of the first compressed artificial intelligence model, or any combination thereof.

14. The electronic device as claimed in claim 12, further comprising: include: monitor, The processor is further configured to, based on the fact that multiple compressed artificial intelligence models obtained by compressing the first artificial intelligence model using multiple methods do not satisfy the predetermined condition, control the display to display a message notifying that the predetermined condition is not satisfied.

15. The electronic device of claim 11, wherein the processor is further configured to, based on the available memory size of the second memory being changed, control to obtain a third compressed artificial intelligence model by compressing the first artificial intelligence model based on the changed available memory size, and load the third compressed artificial intelligence model.

Citation Information

Patent Citations

  • Method and apparatus for compressing memory of electronic device

    CN104737137A

  • Robot control system based on artificial intelligence design

    CN205450754U

  • Buffer memory managing device, buffer memory managing method and computer readable recording medium in which program to make computer implement the same method is recorded

    JP2002055874A