Electronic device performing calculation based on artificial intelligence model and operating method thereof

Abstract

According to various embodiments, an electronic device is provided, including a memory and at least one processor, wherein the at least one processor is configured to: obtain weight values reflecting noise values by applying the noise values to weight values of at least a portion of a plurality of layers included in an artificial intelligence model stored in the electronic device; when an event for executing the artificial intelligence model is identified, obtain calculation data based on a calculation of data input to the at least a portion of the plurality of layers by using the weight values reflecting the noise values; and obtain output data based on the obtained calculation data and the reflected noise values.

Description

Technical Field

[0001] This disclosure relates to an electronic device for optimizing artificial intelligence models and a method of operating the same. Background Technology

[0002] Portable digital communication devices have become an essential element of modern life. Consumers expect to receive a variety of high-quality services anytime, anywhere by using portable digital communication devices.

[0003] Recently, artificial intelligence models trained based on artificial intelligence learning algorithms have been stored in portable digital communication devices, and various types of data obtained by using the artificial intelligence models trained by portable digital communication devices have been processed, enabling the provision of a variety of high-quality services.

[0004] However, as the resources required to manage artificial intelligence models increase, the need for technologies to optimize the computational processes of artificial intelligence models in portable digital communication devices also increases. Summary of the Invention

[0005] Technical issues Electronic devices can store multiple pre-trained artificial intelligence models (e.g., deep learning models or machine learning models). A pre-trained AI model can include multiple layers and can include at least one parameter (e.g., weight values, activation functions, and biases) for computation on data input to each of the multiple layers. The electronic device can process data based on the execution of the AI ​​models, using at least one parameter included in each AI model, to provide various types of services to users. However, when AI models are stored in and executed by the electronic device, there is a possibility that information about the AI ​​models may be exposed externally, potentially leading to security issues. Furthermore, for security enhancement, when AI models are stored and executed in a secure area within the electronic device, computational performance issues may occur due to the lack of computing power allocated to that secure area.

[0006] According to various embodiments, electronic devices and methods of operation thereof can allow pre-trained artificial intelligence models to be stored and executed in a secure environment, thereby enhancing the security of the artificial intelligence models stored in the electronic device. Additionally, according to various embodiments, when performing at least a portion of the computation based on the artificial intelligence model stored in the secure environment (e.g., computation based on at least some layers), the electronic devices and methods of operation thereof can use computing power allocated in a normal environment and prevent the values ​​used for computation from being exposed to the outside (e.g., by applying noise values), thereby enhancing computational performance and maintaining enhanced security.

[0007] Technical solutions According to various embodiments, an electronic device may be provided, the electronic device including a memory and at least one processor, wherein the at least one processor is configured to: obtain a weight value with applied noise value by applying a noise value to weight values ​​of at least a portion of a plurality of layers included in an artificial intelligence model stored in the electronic device; when an event for executing the artificial intelligence model is identified, obtain computational data based on calculations of data input to the at least portion of the plurality of layers using the weight value with applied noise value; and obtain output data based on the obtained computational data and the applied noise value.

[0008] According to various embodiments, an operating method for an electronic device may be provided, the method comprising: obtaining a weight value with applied noise value by applying noise value to weight values ​​of at least a portion of multiple layers included in an artificial intelligence model stored in the electronic device; when an event for executing the artificial intelligence model is identified, obtaining computational data based on calculations of data input to the at least a portion of the multiple layers using the weight value with applied noise value; and obtaining output data based on the obtained computational data and the applied noise value.

[0009] According to various embodiments, an electronic device can be provided, the electronic device including a memory and at least one processor, wherein the at least one processor is configured to: in a trusted execution environment, obtain weight values ​​with applied noise values ​​by applying noise values ​​to weight values ​​of at least a portion of multiple layers included in an artificial intelligence model stored in the electronic device; when an event for executing the artificial intelligence model is identified, in a rich execution environment, obtain computational data based on computation of data input to at least a portion of the multiple layers using the weight values ​​with applied noise values; based on the obtained computational data, change the state of the electronic device from the rich execution environment to the trusted execution environment; and in the trusted execution environment, obtain output data based on the obtained computational data and the applied noise values.

[0010] The technical solutions according to various embodiments are not limited to the solutions described above, and those skilled in the art to which this disclosure pertains can clearly understand other solutions not mentioned from the specification and drawings.

[0011] Beneficial effects According to various embodiments, an electronic device and its operating method can be provided, wherein the electronic device and its operating method can store and execute pre-stored artificial intelligence models in a secure environment, thereby enhancing the security of the artificial intelligence models stored in the electronic device.

[0012] Additionally, according to various embodiments, when performing at least a portion of computation based on an artificial intelligence model stored in a secure environment (e.g., computation based on at least some layers), the electronic device and its operating method can use computing devices allocated in a normal environment and prevent the values ​​used for computation from being exposed to the outside (e.g., by applying noise values), thereby enhancing computing performance and maintaining enhanced security. Attached Figure Description

[0013] Figure 1 This is a block diagram of an electronic device in a network environment according to various embodiments; Figure 2 Examples of configurations of electronic devices according to various embodiments are shown; Figure 3a Examples of execution environments for electronic devices including a single processor, according to various embodiments, are shown; Figure 3b This illustrates another example of the execution environment of an electronic device including a single processor according to various embodiments; Figure 3c Examples of execution environments for electronic devices including multiple processors according to various embodiments are shown; Figure 4a Examples of artificial neural networks according to various embodiments are shown; Figure 4b Another example of an artificial neural network according to various embodiments is shown; Figure 5a This is a flowchart illustrating examples of the operation of an electronic device according to various embodiments; Figure 5b Examples are shown of operations for selecting an artificial intelligence model based on the execution of an electronic device-based application according to various embodiments; Figure 6a Examples of pre-computational operations of electronic devices in a trusted execution environment according to various embodiments are shown; Figure 6b Examples of values ​​stored in an electronic device according to pre-calculated operations are shown according to various embodiments; Figure 7 Examples are shown, according to various embodiments, of performing computational operations based on an artificial intelligence model using pre-calculated values ​​from an electronic device; Figure 8 This is a flowchart illustrating examples of the operation of an electronic device according to various embodiments; Figure 9 Examples of idle and active states of electronic devices according to various embodiments are shown; Figure 10 This is a flowchart illustrating examples of the operation of an electronic device according to various embodiments; Figure 11 Examples of operations are shown, according to various embodiments, of performing pre-computation operations on a selected artificial intelligence model in the idle state of an electronic device; Figure 12 This is a flowchart illustrating examples of the operation of an electronic device according to various embodiments; Figure 13a Examples of operations performing computations on layers of an artificial intelligence model for an electronic device, according to various embodiments, are shown, wherein, for this layer, no pre-computation operation has been completed (or no pre-computation operation has been performed); and Figure 13b Examples of operations for performing computations on layers of an artificial intelligence model for an electronic device, according to various embodiments, are shown, wherein pre-computation operations have been performed for that layer. Detailed Implementation

[0014] Figure 1 This is a block diagram illustrating an electronic device 101 in a network environment 100 according to various embodiments. (Refer to...) Figure 1 In network environment 100, electronic device 101 can communicate with electronic device 102 via a first network 198 (e.g., a short-range wireless communication network), or with at least one of electronic device 104 or server 108 via a second network 199 (e.g., a long-range wireless communication network). According to an embodiment, electronic device 101 can communicate with electronic device 104 via server 108. According to an embodiment, electronic device 101 may include a processor 120, memory 130, input module 150, sound output module 155, display module 160, audio module 170, sensor module 176, interface 177, connection terminal 178, haptic module 179, camera module 180, power management module 188, battery 189, communication module 190, user identification module (SIM) 196, or antenna module 197. In some embodiments, at least one of the above components (e.g., connection terminal 178) may be omitted from electronic device 101, or one or more other components may be added to electronic device 101. In some embodiments, some of the components described above (e.g., sensor module 176, camera module 180, or antenna module 197) may be implemented as a single integrated component (e.g., display module 160).

[0015] Processor 120 may run software (e.g., program 140) to control at least one other component (e.g., hardware or software component) of electronic device 101 connected to processor 120, and may perform various data processing or calculations. According to one embodiment, as at least part of the data processing or calculation, processor 120 may store commands or data received from another component (e.g., sensor module 176 or communication module 190) in volatile memory 132, process the commands or data stored in volatile memory 132, and store the resulting data in non-volatile memory 134. According to embodiments, processor 120 may include a main processor 121 (e.g., central processing unit (CPU) or application processor (AP)) or an auxiliary processor 123 (e.g., graphics processing unit (GPU), neural processing unit (NPU), image signal processor (ISP), sensor central processor, or communication processor (CP)) that is operationally independent of or combined with the main processor 121. For example, when electronic device 101 includes a main processor 121 and an auxiliary processor 123, the auxiliary processor 123 may be adapted to consume less power than the main processor 121, or to be dedicated to a specific function. The auxiliary processor 123 may be implemented separately from the main processor 121, or may be implemented as part of the main processor 121.

[0016] When the main processor 121 is inactive (e.g., in sleep) state, the auxiliary processor 123 (rather than the main processor 121) can control at least some of the functions or states associated with at least one component of the electronic device 101 (e.g., display module 160, sensor module 176, or communication module 190), or when the main processor 121 is active (e.g., running an application), the auxiliary processor 123 can work with the main processor 121 to control at least some of the functions or states associated with at least one component of the electronic device 101 (e.g., display module 160, sensor module 176, or communication module 190). According to embodiments, the auxiliary processor 123 (e.g., an image signal processor or a communication processor) may be implemented as part of another component (e.g., camera module 180 or communication module 190) functionally associated with the auxiliary processor 123. According to embodiments, the auxiliary processor 123 (e.g., a neural processing unit) may include hardware architecture dedicated to artificial intelligence model processing. Artificial intelligence models can be generated through machine learning. For example, such learning can be performed via electronic device 101 where artificial intelligence is performed or via a separate server (e.g., server 108). The learning algorithm may include, but is not limited to, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning. The artificial intelligence model may include multiple layers of artificial neural networks. The artificial neural network may be a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), or a deep Q-network, or a combination of two or more thereof, but is not limited thereto. Additionally or optionally, the artificial intelligence model may include software structures in addition to hardware structures.

[0017] Memory 130 may store various data used by at least one component of electronic device 101 (e.g., processor 120 or sensor module 176). The various data may include, for example, software (e.g., program 140) and input or output data for commands associated with it. Memory 130 may include volatile memory 132 or non-volatile memory 134.

[0018] The program 140 may be stored as software in the memory 130, and the program 140 may include, for example, an operating system (OS) 142, middleware 144, or application 146.

[0019] The input module 150 can receive commands or data from outside the electronic device 101 (e.g., a user) that will be used by other components of the electronic device 101 (e.g., processor 120). The input module 150 may include, for example, a microphone, mouse, keyboard, keys (e.g., buttons), or digital pen (e.g., stylus).

[0020] The sound output module 155 can output sound signals to the outside of the electronic device 101. The sound output module 155 may include, for example, a speaker or a receiver. The speaker can be used for general purposes such as playing multimedia or playing records. The receiver can be used to receive incoming calls. According to an embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

[0021] Display module 160 can visually provide information to the outside of electronic device 101 (e.g., to a user). Display module 160 may include, for example, a display, a holographic device, or a projector, and control circuitry for controlling a respective one of the display, holographic device, and projector. According to an embodiment, display module 160 may include a touch sensor adapted to detect touch or a pressure sensor adapted to measure the intensity of the force caused by touch.

[0022] The audio module 170 can convert sound into electrical signals and vice versa. According to an embodiment, the audio module 170 can obtain sound via the input module 150, or output sound via the sound output module 155 or headphones of an external electronic device (e.g., electronic device 102) that is directly (e.g., wired) or wirelessly connected to the electronic device 101.

[0023] Sensor module 176 can detect the operating state of electronic device 101 (e.g., power or temperature) or the environmental state outside electronic device 101 (e.g., user state), and then generate an electrical signal or data value corresponding to the detected state. According to embodiments, sensor module 176 may include, for example, a gesture sensor, gyroscope sensor, atmospheric pressure sensor, magnetic sensor, accelerometer, grip sensor, proximity sensor, color sensor, infrared (IR) sensor, biometric sensor, temperature sensor, humidity sensor, or illuminance sensor.

[0024] Interface 177 may support one or more specific protocols used to enable electronic device 101 to connect directly (e.g., wired) or wirelessly to external electronic devices (e.g., electronic device 102). According to embodiments, interface 177 may include, for example, a High Definition Multimedia Interface (HDMI), a Universal Serial Bus (USB) interface, a Secure Digital Card (SD) interface, or an audio interface.

[0025] Connection 178 may include a connector, through which electronic device 101 may be physically connected to an external electronic device (e.g., electronic device 102). According to embodiments, connection 178 may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0026] The haptic module 179 can convert electrical signals into mechanical stimuli (e.g., vibration or motion) or electrical stimuli that can be recognized by a user through his touch or kinesthesia. According to an embodiment, the haptic module 179 may include, for example, a motor, a piezoelectric element, or an electrical stimulator.

[0027] Camera module 180 can capture still or moving images. According to an embodiment, camera module 180 may include one or more lenses, an image sensor, an image signal processor, or a flash.

[0028] The power management module 188 manages the power supply to the electronic device 101. According to an embodiment, the power management module 188 may be implemented as at least part of, for example, a power management integrated circuit (PMIC).

[0029] Battery 189 can power at least one component of electronic device 101. According to an embodiment, battery 189 may include, for example, a non-rechargeable primary battery, a rechargeable rechargeable battery, or a fuel cell.

[0030] Communication module 190 can support the establishment of a direct (e.g., wired) or wireless communication channel between electronic device 101 and external electronic devices (e.g., electronic device 102, electronic device 104, or server 108), and perform communication via the established communication channel. Communication module 190 may include one or more communication processors capable of operating independently of processor 120 (e.g., application processor (AP)) and support direct (e.g., wired) or wireless communication. According to embodiments, communication module 190 may include wireless communication module 192 (e.g., cellular communication module, short-range wireless communication module, or Global Navigation Satellite System (GNSS) communication module) or wired communication module 194 (e.g., local area network (LAN) communication module or power line communication (PLC) module). One of these communication modules can communicate with an external electronic device via a first network 198 (e.g., a short-range communication network such as Bluetooth, Wi-Fi Direct, or Infrared Data Association (IrDA)) or a second network 199 (e.g., a long-range communication network such as a traditional cellular network, 5G network, next-generation communication network, the Internet, or a computer network (e.g., a LAN or a wide area network (WAN))). These various types of communication modules can be implemented as a single component (e.g., a single chip) or as multiple components separate from each other (e.g., multiple chips). The wireless communication module 192 can identify and verify the electronic device 101 in the communication network (such as the first network 198 or the second network 199) using user information (e.g., the International Mobile Subscriber Identity (IMSI)) stored in the user identification module 196.

[0031] Wireless communication module 192 can support 5G networks following 4G networks and next-generation communication technologies (such as new radio (NR) access technologies). NR access technologies can support enhanced mobile broadband (eMBB), massive machine-type communication (mMTC), or ultra-reliable low-latency communication (URLLC). Wireless communication module 192 can support high-frequency bands (e.g., millimeter-wave bands) to achieve, for example, high data transmission rates. Wireless communication module 192 can support various technologies used to ensure performance in high-frequency bands, such as, for example, beamforming, massive MIMO, full-dimensional MIMO (FD-MIMO), array antennas, analog beamforming, or massive antennas. Wireless communication module 192 can support various requirements specified in electronic device 101, external electronic devices (e.g., electronic device 104), or network systems (e.g., second network 199). According to an embodiment, the wireless communication module 192 may support peak data rates (e.g., 20 Gbps or greater) for implementing eMBB, lost coverage (e.g., 164 dB or less) for implementing mMTC, or U-plane latency (e.g., 0.5 ms or less for each of the downlink (DL) and uplink (UL), or 1 ms or less round trip) for implementing URLLC.

[0032] Antenna module 197 can transmit or receive signals or power to or from the exterior of electronic device 101 (e.g., external electronic device). According to an embodiment, antenna module 197 may include an antenna comprising a radiating element formed of a conductive material or conductive pattern formed in or on a substrate (e.g., a printed circuit board (PCB)). According to an embodiment, antenna module 197 may include multiple antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication scheme used in a communication network (such as a first network 198 or a second network 199) can be selected from the multiple antennas by, for example, communication module 190 (e.g., wireless communication module 192). Signals or power can then be transmitted or received between communication module 190 and the external electronic device via the selected at least one antenna. According to an embodiment, additional components besides the radiating element (e.g., a radio frequency integrated circuit (RFIC)) may be additionally incorporated into antenna module 197.

[0033] According to various embodiments, antenna module 197 may form a millimeter-wave antenna module. According to embodiments, the millimeter-wave antenna module may include a printed circuit board, a radio frequency integrated circuit (RFIC), and multiple antennas (e.g., an array antenna), wherein the RFIC is disposed on or adjacent to a first surface (e.g., a bottom surface) of the printed circuit board and is capable of supporting a specified high-frequency band (e.g., a millimeter-wave band), and the multiple antennas are disposed on or adjacent to a second surface (e.g., a top surface or a side surface) of the printed circuit board and are capable of transmitting or receiving signals in the specified high-frequency band.

[0034] At least some of the aforementioned components can be interconnected and communicate signals (e.g., commands or data) between them via an inter-peripheral communication scheme (e.g., bus, general purpose input / output (GPIO), serial peripheral interface (SPI), or mobile industrial processor interface (MIPI)).

[0035] According to an embodiment, commands or data can be sent or received between electronic device 101 and external electronic device 104 via server 108 connected to a second network 199. Each of electronic device 102 or electronic device 104 can be a device of the same type as electronic device 101, or a device of a different type. According to an embodiment, all or some operations that would be performed on electronic device 101 can be performed on one or more of external electronic devices 102, external electronic devices 104, or server 108. For example, if electronic device 101 is required to automatically perform a function or service, or is required to perform a function or service in response to a request from a user or another device, electronic device 101 may request the one or more external electronic devices to perform at least a portion of the function or service, instead of running the function or service, or electronic device 101 may request the one or more external electronic devices to perform at least a portion of the function or service in addition to running the function or service. Upon receiving the request, one or more external electronic devices may perform at least a portion of the requested function or service, or perform additional functions or services related to the request, and transmit the result of the execution to electronic device 101. Electronic device 101 may provide the result as at least a partial response to the request, with or without further processing of the result. For this purpose, technologies such as cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing may be used. Electronic device 101 may use, for example, distributed computing or mobile edge computing to provide ultra-low latency services. In another embodiment, external electronic device 104 may include an Internet of Things (IoT) device. Server 108 may be an intelligent server using machine learning and / or neural networks. According to embodiments, external electronic device 104 or server 108 may be included in a second network 199. Electronic device 101 may be applied to intelligent services based on 5G communication technology or IoT-related technologies (e.g., smart homes, smart cities, smart cars, or healthcare).

[0036] The electronic device 101 according to various embodiments can be one of various types of electronic devices. The electronic device 101 may include, for example, a portable communication device (e.g., a smartphone), a computer device, a portable multimedia device, a portable medical device, a camera, an electronic device, or a home appliance. According to embodiments of this disclosure, the electronic device 101 is not limited to the examples described above.

[0037] It should be understood that the various embodiments of this disclosure and the terminology used therein are not intended to limit the technical features set forth herein to the specific embodiments, but rather to include various changes, equivalents, or substitutions to the respective embodiments. In the description of the drawings, similar reference numerals may be used to refer to similar or related elements. It will be understood that nouns in the singular form corresponding to terms may include one or more things unless the relevant context clearly indicates otherwise. As used herein, each of the phrases such as “A or B,” “at least one of A and B,” “at least one of A or B,” “A, B, or C,” “at least one of A, B, and C,” and “at least one of A, B, or C” may include any one or all possible combinations of the items enumerated together with the corresponding phrase among the plurality of phrases. As used herein, terms such as “first” and “second” or “first” and “second” may be used to simply distinguish the respective component from another component and do not limit the component in other respects (e.g., importance or order). It will be understood that, whether the terms “operably” or “communically” are used or not, if an element (e.g., a first element) is referred to as “combined with another element (e.g., a second element),” “combined to another element (e.g., a second element),” “connected to another element (e.g., a second element),” or “connected to another element (e.g., a second element)”, it means that the element can be directly (e.g., wiredly) connected to the other element, wirelessly connected to the other element, or connected to the other element via a third element.

[0038] As used in connection with various embodiments of this disclosure, the term "module" may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with other terms such as "logic," "logic block," "part," or "circuit." A module may be a single integrated component adapted to perform one or more functions, or the smallest unit or part of such a single integrated component. For example, according to embodiments, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0039] The various embodiments set forth herein can be implemented as software (e.g., program 140) containing one or more instructions readable by a machine (e.g., electronic device 101) stored in a storage medium (e.g., internal memory 136 or external memory 138). For example, under the control of a processor, the processor (e.g., processor 120) of the machine (e.g., electronic device 101) can invoke and execute at least one of the one or more instructions stored in the storage medium, with or without the use of one or more other components. This enables the machine to operate to perform at least one function according to the invoked at least one instruction. The one or more instructions may include code generated by a compiler or code executable by an interpreter. Machine-readable storage media can be provided in the form of non-transitory storage media. While the term "non-transitory" only means that the storage medium is a tangible device and does not include signals (e.g., electromagnetic waves), this term does not distinguish between data being stored semi-permanently in the storage medium and data being temporarily stored in the storage medium.

[0040] According to embodiments, methods according to various embodiments of this disclosure may be included and provided in a computer program product. The computer program product can be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., a compact disk read-only memory (CD-ROM)) or via an app store (e.g., the Play Store). TM The computer program product may be published online (e.g., downloaded or uploaded), or may be distributed directly between two user devices (e.g., smartphones) (e.g., downloaded or uploaded). If published online, at least a portion of the computer program product may be temporarily generated, or at least a portion of the computer program product may be temporarily stored in a machine-readable storage medium (such as the memory of a manufacturer's server, an app store's server, or a forwarding server).

[0041] According to various embodiments, each of the above-described components (e.g., a module or program) may include a single entity or multiple entities, and some of the multiple entities may be separately disposed in different components. According to various embodiments, one or more of the above-described components may be omitted, or one or more other components may be added. Optionally or additionally, multiple components (e.g., modules or programs) may be integrated into a single component. In this case, according to various embodiments, the integrated component may still perform the one or more functions of each of the multiple components in the same or similar manner as the corresponding component of the multiple components performed one or more functions before integration. According to various embodiments, the operations performed by a module, program, or other component may be performed sequentially, in parallel, repeatedly, or heuristically, or one or more of the operations may be run in a different order or omitted, or one or more other operations may be added.

[0042] In the following text, electronic devices according to various embodiments (e.g., Figure 1 Example of the configuration of electronic device 101). The electronic device described below can be as follows: Figure 1 As shown in the above-described electronic device 101, redundant descriptions will be omitted.

[0043] Figure 2 Examples of configurations of electronic device 101 according to various embodiments are shown. However, this disclosure is not limited to... Figure 2 The components shown, and the electronic device 101 can be implemented to include more than Figure 2 The number of elements shown may be more or less. See below for reference. Figures 3a to 3c as well as Figure 4a and Figure 4b describe Figure 2 .

[0044] Figure 3a Examples of execution environments for electronic devices 101 including a single processor 250 according to various embodiments are shown. Figure 3b Another example of the execution environment of an electronic device 101 including a single processor 250 according to various embodiments is shown. Figure 3c Examples of the execution environment of an electronic device 101, comprising multiple processors (e.g., a first processor 250a and a second processor 250b), according to various embodiments are shown. Figures 3a to 3c In the middle, the left and right sides of the reference dashed line can respectively include different execution environments (e.g., Rich Execution Environment (REE) 310 and Trusted Execution Environment (TEE)). Figure 4a Examples of artificial neural networks according to various embodiments are shown. Figure 4bAnother example of an artificial neural network according to various embodiments is shown.

[0045] According to various embodiments, refer to Figure 2 The electronic device 101 may include a data acquisition device 210 (which includes a camera 211, a microphone 213, and communication circuitry 215), multiple computing devices 220, a memory 230 for storing multiple artificial intelligence models 231, multiple applications 233, and modules 240 (e.g., a noise addition module 241, a computation execution module 243, and an output data acquisition module 245), and a processor 250. Each element included in the electronic device 101 is described below.

[0046] The following describes a data acquisition apparatus 210 according to various embodiments. The data acquisition apparatus 210 can be understood as a logical concept for classifying the means for acquiring content among the means included in the electronic device 101. In addition to the camera 211, microphone 213, and communication circuitry 215, the data acquisition apparatus 210 may also include various types of means (e.g., various types of sensors and touchscreens) for acquiring the various types of content described below.

[0047] According to various embodiments, the data acquisition device 210 can acquire various types of data (or content) that will be processed based on the artificial intelligence model 231 described below. These various types of data may include electronic documents and media data, such as image, video, and audio data, and this disclosure is not limited thereto, and may also include various types of electronic data (e.g., software and sensor values) that can be electronically analyzed by the artificial intelligence model. According to embodiments, the data acquisition device 210 is driven by executing and / or driving applications, programs, and / or processes installed (or stored) in the electronic device 101 to acquire various types of data. For example, when executing and / or driving a camera application, the electronic device 101 may drive the camera 211 (e.g., perform operations to control the readout of the image sensor) to acquire images and / or video as data. In another example, when executing and / or driving a recording application, the electronic device 101 may drive the microphone 213 to acquire audio data such as ambient sounds and / or user speech as data. In another example, when executing and / or driving a web-based application, electronic device 101 can configure a communication connection with a media server using communication circuitry 215 and acquire media data such as image, video, and audio data. Examples of each data acquisition device 210 are described below.

[0048] According to various embodiments, camera 211 can capture still images (or pictures) or moving images. According to embodiments, at least one camera 211 may include one or more lenses, image sensors, image signal processors, or flashes. According to embodiments, electronic device 101 may include cameras 211 with different attributes or functions (or purposes). For example, at least one camera 211 may include cameras with different angles. For example, the angles may include 114 to 94 degrees ultra-wide-angle, wide-angle, 84 to 63 degrees normal lens, 28 to 8 degrees telephoto, and 6 to 3 degrees super telephoto. In another example, at least one camera 211 may include at least one front-facing camera disposed on the front surface of electronic device 101 for capturing images and / or recording videos, and at least one rear-facing camera disposed on the rear surface of electronic device 101 for capturing images and / or recording videos.

[0049] According to various embodiments, microphone 213 can receive sound from outside the electronic device 101. For example, the electronic device 101 (e.g., processor 250) can drive microphone 213 to receive sound generated from outside the device. The sound generated from outside the device can include speech (or utterance) from a speaker (e.g., a user and / or another speaker (or other person)), residential noise, and ambient (background) noise. According to embodiments, microphone 213 can include multiple microphones 213. The electronic device 101 (e.g., processor 250) can perform beamforming to receive sound generated in a specified direction of the electronic device 101 from the sound received using the multiple microphones 213. Based on the received sound, the sound acquired in the specified direction can be defined as a sub-sound. The multiple microphones 213 can be arranged in the electronic device 101 to be spaced apart from each other by a predetermined distance, and signal processing can be performed on the sound received through each microphone 213 by the spacing distance and the phase or time associated with the direction in which the sound is to be acquired, in order to acquire the sub-sound. Beamforming techniques are known in the art, and therefore a detailed description will be omitted.

[0050] According to various embodiments, the communication circuit 215 can establish a communication connection with an external electronic device (e.g., another electronic device or a server) and send and / or receive data according to various types of communication schemes. As mentioned above, the communication scheme can be implemented through a direct communication connection configured therein, such as Bluetooth and Wi-Fi Direct, but this disclosure is not limited thereto and may include communication schemes using access points (e.g., Wi-Fi communication) or communication schemes using cellular communication with base stations (e.g., 3G, 4G / LTE, and 5G). The communication circuit 215 can be implemented as follows: Figure 1 The aforementioned communication module 190 will therefore omit redundant descriptions.

[0051] In the following text, a plurality of computing devices 220 are described.

[0052] According to various embodiments, each of the plurality of computing devices 220 may be configured to perform computations (e.g., matrix computations (e.g., matrix multiplication)) based on an artificial intelligence model 231 (or associated with it) stored in the electronic device 101. For example, the plurality of computing devices 220 may include, but are not limited to, at least one of an application processor (AP) (not shown), a central processing unit (CPU) (not shown), a graphics processing unit (GPU) 221, a display processing unit (DPU) 223, or a neural processing unit (NPU) (not shown), and may include various types of processors for computation. In the specification, the plurality of cores included in a processor may be understood as processor 250. For example, when a digital signal processor (DSP) is implemented to include multiple cores, each of the plurality of cores may be understood as processor 250. Computations associated with the artificial intelligence model may include computations (e.g., matrix computations, bias computations, and activation function computations) based on the artificial intelligence model 231, which includes pre-trained layers. Although described below, each of the plurality of computing devices 220 can be configured to perform computation in a specific execution environment (e.g., a rich execution environment (REE) 310 or a trusted execution environment (TEE) 320) (or execution mode or processor 250). For example, a rich execution environment (REE) can refer to a general-purpose execution environment with a low level of security, and a trusted execution environment (TEE) can refer to a secure execution environment with a high level of security. A secure execution environment can, for example, store data requiring a relatively high level of security and perform related operations in a secure environment. A trusted execution environment can operate in a secure domain, for example, by dividing an application processor or memory into a general domain and a secure domain, and software or hardware requiring security can operate only in the secure domain. For example, most of the computing devices 220 can be configured to perform computation based on an artificial intelligence model in the rich execution environment 310, and the remaining computing devices 220 can be configured to perform computation in the trusted execution environment 320. The operation of performing computation based on an artificial intelligence model by the plurality of computing devices 220 can be performed in the background, but is not limited to the background, and can also be performed in the foreground.

[0053] According to various embodiments, values ​​associated with the available computing power of each of the plurality of computing devices 220 can be preset. For example, the values ​​associated with computing power may include, but are not limited to, 32-bit values, 16-bit values, 8-bit values, and 4-bit values, and can be configured with various values, and the range of numbers and types used for calculation can be determined based on the configured values. For example, when the computing device is configured with a 32-bit value for the calculation, the computing device can calculate the weights included in the artificial intelligence model 231 in floating-point units (32-bit floating-point) with an 8-bit exponent and a 24-bit mantissa. The types and ranges of numbers that can be represented based on other calculation values ​​are known in the art, and therefore a detailed description will be omitted.

[0054] In the following text, an artificial intelligence model 231 according to various embodiments is described.

[0055] According to various embodiments, each of the plurality of artificial intelligence models 231, which has been pre-trained based on a specified type of learning algorithm, may be a pre-implemented artificial intelligence model that receives and computes various types of data (or content) and outputs (or acquires) result data. For example, in electronic device 101, learning is performed based on machine learning algorithms or deep learning algorithms to output specific types of result data as output data by using specified types of data as input data, and a plurality of artificial intelligence models 231 (e.g., machine learning models and deep learning models) are generated, such that the generated plurality of artificial intelligence models 231 can be stored in electronic device 101, or an artificial intelligence model 231 that has been trained by an external electronic device (e.g., an external server) can be sent to electronic device 101 and stored in electronic device 101. According to embodiments, when an artificial intelligence model is received (or downloaded) from an external server to electronic device 101, the external server may correspond to a third-party server for creating applications or a third-party management server for generating applications, and the artificial intelligence model corresponding to the functions to be provided by the application may be registered in the external server along with the application. Therefore, the application and the corresponding artificial intelligence model can be sent together from an external server to electronic device 101 (e.g., downloaded to electronic device 101), but this disclosure is not limited thereto. Machine learning algorithms may include, but are not limited to, supervised algorithms such as linear regression and logistic regression, unsupervised algorithms such as clustering, visualization and dimensionality reduction, and association rule learning, as well as reinforcement algorithms, and deep learning algorithms may include artificial neural networks (ANNs), deep neural networks (DNNs), and convolutional neural networks (CNNs), and may also include various learning algorithms. According to embodiments, each of the plurality of artificial intelligence models 231 can be configured to be executed using a specific computing device among the plurality of computing devices 220. Therefore, computations based on each of the plurality of artificial intelligence models 231 (e.g., calculating weights in 32-bit floating-point form) can be performed based on values ​​associated with computing power pre-configured for use by a specific computing device. However, this disclosure is not limited thereto, and the computing device for executing each of the plurality of artificial intelligence models 231 may be randomly selected from the plurality of computing devices 220.

[0056] According to various embodiments, the learned artificial intelligence model 231 may include one or more parameters (e.g., at least one of weight values, activation functions, or biases) for calculating input data, and output data may be obtained based on the parameters by calculating the input data. For example, as Figure 4aAs shown, the learned artificial intelligence model 231 may include multiple layers (e.g., a first layer (layer 1) and a second layer (layer 2)), and for each of the multiple layers, it may include at least one of the following: weight values ​​w1-1, w2-1, and w3-1 (or weight matrix (W)), activation functions (e.g., sigmoid and ReLU functions), or biases (or bias vectors (B)) for computing input data (e.g., X1 vector) for each input of the multiple layers. Layers (e.g., the first layer (layer 1) and the second layer (layer 2)) can be understood as logical concepts stored by classifying the values ​​(e.g., weight values, activation functions, and biases) used for computation. For example, refer to... Figure 4a The artificial intelligence model 231 can be fed into the first layer (layer 1) based on input data (X1, X2, and X3), and output data (Y1) can be obtained by multiplying each input data (X1, X2, or X3) with weights configured for each node of the first layer (layer 1) (e.g., w1-1, w2-1, or w3-1), adding the bias (B) to each input data obtained by multiplying the weights, and applying an activation function (δ(x)) to the summed result. Therefore, as shown in [Equation 1] below, with respect to a specific layer (e.g., layer 1) of the artificial intelligence model 231, the electronic device 101 can perform the following matrix calculations, and the values ​​used for the calculations can be stored in the weight matrix and bias matrix for each layer.

[0057] [Equation 1]

[0058] In [Equation 1], n represents the identifier (or sequence) of the layer, Xn represents the data input for each layer, An represents the weight matrix of each layer, Bn represents the bias matrix of each layer, and δ represents the activation function.

[0059] Output data (e.g., the Y1 matrix) calculated from a layer (e.g., the first layer, layer 1) can be input to the next layer (e.g., the second layer, layer 2), and the input data can be calculated based on the weight values, biases, or activation functions of the second layer (layer 2). Therefore, the sequential calculations of the aforementioned layers of the artificial intelligence model can be performed sequentially until the calculated result data is output in the output layer (not shown). This disclosure is not limited to the above description or illustration, and the bias matrix and / or activation function may not be implemented depending on the implementation purpose. In another example, such as Figure 4bAs shown, when training an artificial intelligence model based on a CNN, the multiple layers can include various types of layers, such as pooling layers and convolutional layers for computing input data (e.g., X1421) and obtaining result data 423. This disclosure is not limited to the above description, and the artificial intelligence model 231 can be implemented according to techniques known in the art; therefore, detailed descriptions will be omitted.

[0060] In the following description, a processor 250 according to various embodiments is described. For ease of description, the processor 250 described and / or shown below is distinguished from the plurality of computing devices 220 described above, but the processor 250 may be one of the plurality of computing devices 220. However, this disclosure is not limited thereto, and the processor 250 may correspond to a processor 250 implemented separately from the plurality of computing devices 220.

[0061] According to various embodiments, processor 250 may include at least one of an application processor (AP), a central processing unit (CPU), a graphics processing unit (GPU), a display processing unit (DPU), or a neural processing unit (NPU). The operations of processor 250 described below can be performed based on the execution of modules 240 (e.g., computation execution module 243, noise addition module 241, and output data acquisition module 245) stored in memory 230. For example, at least a portion of module 240 (e.g., computation execution module 243, noise addition module 241, and output data acquisition module 245) may be implemented (e.g., executed) by software, firmware, or a combination of two or more thereof. For example, modules may be implemented in the form of procedures, routines, instructions, computer code, programs, and applications executable by processor 250. Therefore, when module 240 is executed by processor 250, module 240 can cause processor 250 to perform operations associated with the module (or functions that can be provided by the module). Therefore, in the following, when a particular module performs an operation, this can mean that as the particular module is executed, processor 250 performs the corresponding operation. Optionally, module 240 may be implemented as part of a specific application. Optionally, this disclosure is not limited to the description and / or illustration above, and each module may be implemented as hardware (e.g., a processor or control circuitry) separate from processor 250.

[0062] According to various embodiments, processor 250 can perform operations based on modules that execute in multiple execution environments spaced apart from each other. For example, execution environments that are spaced apart from each other in terms of software can be implemented, or execution environments that are spaced apart from each other in terms of hardware can be implemented. Examples of each of the execution environments are described below.

[0063] The following describes examples of execution environments spaced apart in terms of software (e.g., Figure 3a and Figure 3b ).

[0064] According to various embodiments, such as in Figure 3a and Figure 3b As shown, electronic device 101 can be implemented to perform operations (or functions) based on multiple execution environments that are software-separated (or independent) from each other. These multiple execution environments may include, but are not limited to, a Rich Execution Environment (REE) 310 and a Trusted Execution Environment (TEE) 320, and may also include various types of execution environments that can be implemented as separate (or independent) from each other. In this case, multiple execution environments can be understood as one type of execution mode for a single processor 250. For example, in the following... Figure 3a In the case of the illustrated electronic device 101, the processor 250 can perform operations in a rich execution environment 310 during a first time interval, or it can perform operations in a trusted execution environment 320 during a second time interval. In this case, the types of hardware and permissions allocated to each execution environment (e.g., rich execution environment 310 and trusted execution environment 320) can be different from each other. In embodiments, a different number of computing devices may be allocated to each execution environment. (See reference...) Figure 3a and Figure 3b Each diagram can indicate, referring to the dashed lines, that the hardware device on the left is assigned to (or driven in) the rich execution environment 310, and the hardware device on the right is assigned to (or driven in) the trusted execution environment 320. For example, refer to... Figure 3a and Figure 3b Compared to the trusted execution environment 320, more computing devices are allocated (or all computing devices are allocated) to the rich execution environment 310, and therefore, the processor 250 can perform operations in the rich execution environment 310 based on a relatively larger number of computing devices. In another embodiment, such as Figure 3a and Figure 3bAs shown, designated regions of memory 230 are allocated to each execution environment (e.g., rich execution environment 310 and trusted execution environment 320). In rich execution environment 310, processor 250 can read data from and write data to a first region 311 in memory (the first region 311 is allocated to rich execution environment 310), and in trusted execution environment 320, processor 250 can read data from and write data to a second region 321 in memory 230 (the second region 321 is allocated to trusted execution environment 320). In another embodiment, the types of security and permissions allocated to the execution environments can be different from each other. For example, the permissions and security of trusted execution environment 320 can be higher than those of rich execution environment 310. For example, in a trusted execution environment 320, processor 250 can access a first region 311 in memory 230 (the first region 311 is allocated to rich execution environment 310) to read and write data therein, but in rich execution environment 310, hardware or information allocated to trusted execution environment 320 (e.g., a second region 321 in memory 230) may be inaccessible. Figure 3a and Figure 3b As shown, the artificial intelligence model 231 can be stored in a region (e.g., a second region 321) of memory 230 in the trusted execution environment 320, and therefore, the processor 250 can access the artificial intelligence model 231 without being in the rich execution environment 310. Thus, information about the artificial intelligence model 231 can be kept from being exposed to the outside.

[0065] According to various embodiments, the processor 250 can perform operations based on modules executing in different execution environments. For example, refer to... Figure 3a and Figure 3b The computation execution module 243 can be configured to be executed in a rich execution environment 310, and the noise addition module 241 and the output data acquisition module 245 can be configured to be executed in a trusted execution environment 320. As described above, due to the relatively high permissions and security in the trusted execution environment 320, the processor 250 can access the data acquired by the output data acquisition module 245 and the noise addition module 241 executed in the trusted execution environment 320 without being in the rich execution environment 310. However, this disclosure is not limited to this, and the execution environment of each module can be implemented in a manner different from the above illustrations and / or descriptions. In embodiments, such as Figure 3a As shown, processor 250 can execute modules in different execution environments based on a configuration used to switch execution environments (e.g., monitor mode 247). For example, as Figure 3aAs shown, the software and hardware of electronic device 101 can be implemented based on an advanced RISC machine (ARM) architecture (ARM architecture), and in this case, the rich execution environment 310 can be defined as the normal world (or normal execution environment), and the trusted execution environment 320 can be defined as the secure world (or secure execution environment). Monitor mode 247 can be executed in the secure world (e.g., trusted execution environment 320), and can be executed when a Security Monitor Call (SMC) command for switching between worlds (e.g., the normal world (e.g., rich execution environment 310) and the secure world (e.g., trusted execution environment 320)) is generated, or when an Interrupt Request (IRQ) or Fast Interrupt Request (FIQ) is generated. For example, as Figure 3a As shown, when an event occurs that triggers an operation based on the noise addition module 241 or the output data acquisition module 245, the processor 250 can switch (or change) the current execution environment to a trusted execution environment 320 via monitor mode 247 to perform the operation based on the noise addition module 241 or the output data acquisition module 245 in the trusted execution environment 320. In another example, when an event occurs that triggers a switch to a rich execution environment 310 via monitor mode 247 and triggers an operation based on the computational execution module 243, the processor 250 can perform the operation based on the computational execution module 243 in the rich execution environment 310. Although not shown, the operating system (OS) executed in each execution environment can be included in the electronic device 101, and each OS can be executed when switching to each execution environment. For example, the processor 250 can execute the OS of the rich execution environment 310 in the rich execution environment 310, and can execute the OS of the trusted execution environment 320 in the trusted execution environment 320. Figure 3a As shown, processor 250 can perform operations based on each execution environment at different time intervals (or at different points in time) that do not overlap (e.g., performing operations in trusted execution environment 320 and then in rich execution environment 310, or performing operations in rich execution environment 310 and then in trusted execution environment 320). However, this disclosure is not limited to the above description, and operations can be performed simultaneously based on two execution environments. In another embodiment, as... Figure 3b As shown, processor 250 can execute modules (e.g., noise addition module 241, computation execution module 243, and output data acquisition module 245) on different virtual OSes running in different execution environments (e.g., rich execution environment 310 and trusted execution environment 320) using virtual OSes running in different execution environments (e.g., rich execution environment 310 and trusted execution environment 320) and a hypervisor 249 managed by the virtual OSes. Figure 3bWhat is achieved in it can be compared with Figure 3a The operation of each module 240 is performed by electronic device 101 in each execution environment in the same manner described above; therefore, redundant descriptions will be omitted. In this case, processor 250 can perform operations based on each execution environment at different time intervals that do not overlap with each other; however, this disclosure is not limited to the above description and can perform operations simultaneously based on two execution environments. Figure 3b The software and hardware of the electronic device 101 shown can be implemented based on Intel's architecture (e.g., SGX (Software Protected Extensions) architecture).

[0066] According to various embodiments, processor 250 can share data through a portion of a region in memory (e.g., a first region 311) allocated to rich execution environment 310. For example, in trusted execution environment 320, processor 250 can write data acquired based on at least a portion of module 240 (e.g., noise addition module 241) into a portion of the first region 311 in memory (the first region 311 is allocated to rich execution environment 310), and in rich execution environment 310, when performing operations based on at least a portion of module 240 (e.g., computation execution module 243), processor 250 can read the data written into the portion of the first region 311. In another example, in the rich execution environment 310, the processor 250 can write (or store) data acquired based on at least a portion of module 240 (e.g., computation execution module 243) into a portion of a first region 311 in memory 230 (the first region is allocated to the rich execution environment 310), and in the trusted execution environment 320, when performing operations based on at least a portion of module 240 (e.g., output data acquisition module 245), the processor 250 can read (or acquire) the data written into the portion of the first region 311. In this case, specific modules (e.g., noise addition module 241, computation execution module 243, and output data acquisition module 245) can be pre-configured as a portion of the first region 311 in reference memory 230. Optionally, this disclosure is not limited to the above description, and data sharing between execution environments can be performed in scenarios where data is transferred from the trusted execution environment 320 to the rich execution environment 310 or vice versa.

[0067] The following describes examples of hardware-based separation of the execution environment (e.g., Figure 3c ).

[0068] According to various embodiments, such as Figure 3cAs shown, electronic device 101 can be implemented to perform operations (or functions) in multiple execution environments based on independently implemented hardware (e.g., memory 230 and computing device). For example, in electronic device 101, different types of hardware (e.g., processors 250a and 250b, memory 230a and 230b, and computing device 220) are assigned to execution environments (e.g., rich execution environment 310 and trusted execution environment 320), and different permissions can be assigned. For example, as Figure 3c As shown, the first processor 250a can execute a separate and independent operating system within the rich execution environment 310, performing operations based on the computing execution module 243 using a computing device, and reading and writing data from the first memory 230a. In another example, within the trusted execution environment 320, the second processor 250b can execute an independent operating system separate from the operating system in the rich execution environment 310, performing operations based on the noise addition module 241 and the output data acquisition module 245, and reading and writing data from the second memory 230b. In this case, the second processor 250b may have permission to access the first memory 230a within the trusted execution environment 320. The descriptions of permissions in the rich execution environment 310 and the trusted execution environment 320 are consistent with... Figure 3a and Figure 3b The description is the same as above, so redundant descriptions will be omitted. In this case, similar to the description above, processors (e.g., first processor 250a and second processor 250b) can perform operations in their respective execution environments within non-overlapping time intervals. However, this disclosure is not limited to the description above, and operations can be performed simultaneously based on two execution environments. Additionally, as mentioned above, each processor (e.g., first processor 250a and second processor 250b) can share data through a portion of a specific region 313a in the first memory 230a, where the specific region 313a is allocated to the rich execution environment 310. However, this disclosure is not limited to this, and data sharing between execution environments can be performed in a scheme where different types of data are transferred between the respective execution environments.

[0069] Hereinafter, examples of each module 240 executed by processor 250 according to various embodiments are described. As described above, computation execution module 243 may be executed in rich execution environment 310, and noise addition module 241 and output data acquisition module 245 may be executed in trusted execution environment 320. This disclosure is not limited to the above description, and modules 240 may be implemented to execute in different execution environments respectively. For example, electronic device 101 may perform computation based on the first layer of a plurality of layers of an artificial intelligence model in trusted execution environment 320 (i.e., computation execution module 243 implemented to perform computation based on the first layer may execute in trusted execution environment 320), which will be described in detail below.

[0070] According to various embodiments, the noise addition module 241 can generate noise values ​​(e.g., weighted noise and input noise) associated with the computation based on the artificial intelligence model 231. For example, the noise addition module 241 can identify some of the multiple layers of the artificial intelligence model 231 and generate weights for the identified layers and weight values ​​to be applied to the data input to the identified layers. (See Figures 5 to...) Figure 7 The description refers to the operation of the noise addition module 241 in identifying some layers. In one embodiment, the noise addition module 241 can generate noise values ​​(hereinafter referred to as weight noise) that will be applied to the weight values ​​(e.g., weight matrices) of some of the multiple layers included in the artificial intelligence model 231, and can apply the generated noise values ​​to the weight values ​​of some of the multiple layers (e.g., multiply the generated noise values ​​by the weight values ​​of some of the multiple layers). The values ​​of the weight noise can be generated randomly, and can be randomly configured within a range of values ​​less than 0.9 or equal to or greater than 1.1. In this case, the values ​​of the weight noise can be configured proportionally to the computing power (e.g., 32-bit or 8-bit) of the computing device used to execute the artificial intelligence model 231. In another embodiment, the noise addition module 241 can generate noise values ​​(e.g., noise vectors) (hereinafter referred to as input noise) that will be applied (e.g., added) to a portion of the data input to a portion of the multiple layers included in the artificial intelligence model 231. The type and number of input noise can be determined according to the type and size of the input data (e.g., the number of vectors), and each value can be randomly generated within the range of values ​​that the input data can have. As described below, the noise addition module 241 can perform operations to pre-generate and apply weighted noise to some layers and to generate input noise, and can pre-store data acquired based on the execution of the operations (e.g., weight values ​​of the layers to which weighted noise is applied, weighted noise, input noise, and data to be shown in Figures 5 to 6). Figure 7 (The noise subtraction matrix described above). This disclosure is not limited to the above description, and the noise addition module 241 can generate noise values ​​that will be applied to the activation function and / or bias vector.

[0071] According to various embodiments, the computation execution module 243 can perform computations based on the artificial intelligence model 231. For example, the computation execution module 243 can perform computations (e.g., matrix multiplication) based on the weight values ​​(e.g., weight matrix) of the layers of the artificial intelligence model 231 to which the noise addition module 241 applies noise values ​​and the input data (e.g., input data matrix) input to the layers to which the noise values ​​are applied, in order to obtain output data. The computations are not limited to those described above and can include various types of computations other than matrix multiplication.

[0072] According to various embodiments, the output data acquisition module 245 can acquire output data by subtracting noise values ​​from the output data acquired by the computation execution module 243. Additionally, when a bias exists in the layer where the output data is computed, the output data acquisition module 245 can add the bias to the acquired output data, and / or when an activation function exists in the layer, the output data acquisition module 245 can further perform a computation to input the acquired output data into the activation function. In this case, to reduce computational load, when performing computation on each layer, the processor 250 can pre-obtain and pre-store the value computed from the output data acquired by the computation execution module 243 (e.g., the sum of the added bias and the subtracted noise values) as a value (or a matrix). For example, the processor 250 (e.g., the noise addition module 241) can pre-obtain and pre-store the noise subtraction matrix (Tn), which will be described below. Subsequently, the output data acquisition module 245 can apply the pre-obtained noise subtraction matrix (Tn) to the computation data acquired by the computation execution module 243 and subtract the weight noise in order to obtain output data with noise removed with a relatively small amount of computation.

[0073] Hereinafter, embodiments of the operation of the electronic device 101 according to various embodiments are described.

[0074] According to various embodiments, electronic device 101 can pre-generate noise values ​​associated with computation based on at least a portion of multiple layers of artificial intelligence model 231 within a trusted execution environment 320. Electronic device 101 can apply the pre-generated noise values ​​to weight values ​​of at least a portion of the multiple layers to store these weight values. Then, when executing artificial intelligence model 231, computation is performed in rich execution environment 310 based on the weight values ​​with the pre-stored noise values ​​applied to obtain computational data acquired based on the execution of the computation. Electronic device 101 can obtain output data by subtracting the noise values ​​from the computational data acquired in trusted execution environment 320, and can continuously perform operations by using the acquired output data as input data for the next layer.

[0075] Figure 5aThis is a flowchart 500 illustrating examples of the operation of an electronic device 101 according to various embodiments. Figure 5a The order of operations shown is not limited to the above description, and operations can be performed in various orders. Furthermore, according to various embodiments, operations can be performed in a more... Figure 5a The operations shown may include more or fewer operations. See below for reference. Figure 5b , Figure 6a and Figure 6b as well as Figure 7 describe Figure 5a .

[0076] Figure 5b Examples are shown of operations for selecting an artificial intelligence model by executing an application based on electronic device 101 according to various embodiments. Figure 6a Examples of pre-computation operations in a trusted execution environment of an electronic device 101 according to various embodiments are shown. Figure 6b Examples of values ​​stored in electronic device 101 according to pre-calculated operations are shown according to various embodiments. Figure 7 Examples are shown of performing computational operations based on an artificial intelligence model using pre-calculated values ​​from electronic device 101, according to various embodiments.

[0077] According to various embodiments, in operation 501, electronic device 101 may apply noise values ​​to weight values ​​of at least a portion of multiple layers included in an artificial intelligence model stored in electronic device 101, in order to obtain weight values ​​with applied noise values. For example, as Figure 6a As shown, electronic device 101 (e.g., processor 250) can select at least a portion of layers (n) from multiple layers of artificial intelligence model 231 in a trusted execution environment 320 (operation 601), apply weight noise (zn) 631 to the weight values ​​(e.g., w) (or weight matrix (An)) of the selected layer (n) and generate a weight matrix (An') 633 with the applied noise values ​​(e.g., w') (or weight noise (zn) 631) (operation 603), and generate input noise (Rn) 651 to be applied to the input data (Xn) to be input into the selected layer (n) (operation 605). The order of operations 601, 603, and 605 is not limited to those described above, and operations can be performed in various orders. Operations 601, 603, and 605 can be defined as pre-computation operations. Figure 6a As shown, when the execution environment (or execution mode) of processor 250 is switched to trusted execution environment 320 via FIQ or IRQ for environment switching and monitor mode 247, processor 250 can perform operations (operations 601, 603, and 605). However, this disclosure is not limited to the above description, and can be extended by managing the aforementioned virtual OS (e.g., as...). Figure 3a (and the implementation example shown in 3B) or based on independently provided hardware (e.g., such as...) Figure 3c The operation is performed using the implementation example shown, therefore redundant descriptions will be omitted. Pre-computation operations can be performed in the idle state of electronic device 101, and therefore, when electronic device 101 is in an idle state, the execution environment of processor 250 can be switched to trusted execution environment 320 to perform pre-computation operations. This will be discussed later. Figure 8 and Figure 9 The idle state is described below. According to an embodiment, electronic device 101 (e.g., processor 250) may perform pre-computation operations (operations 601, 603, and 605) for each of a plurality of artificial intelligence models 231 stored in memory 230. In another embodiment, electronic device 101 (e.g., processor 250) may perform pre-computation operations only for some of the plurality of artificial intelligence models 231. For example, the manufacturer of the artificial intelligence model, the manufacturer of the application corresponding to the artificial intelligence model, or the user of the artificial intelligence model (or the user of electronic device 101) may predetermine whether to perform pre-computation operations for the artificial intelligence model, and may pre-store information indicating whether to perform pre-computation operations in electronic device 101. Processor 250 may perform pre-computation operations for some of the plurality of artificial intelligence models 231 based on this information. The pre-computation operations will be described in more detail below.

[0078] According to various embodiments, electronic device 101 (e.g., noise addition module 241) can select a portion (n) of multiple layers of a plurality of artificial intelligence models 231 (e.g., model A). For example, noise addition module 241 can select a portion of the layers remaining after excluding a first layer from the plurality of layers of the artificial intelligence model (e.g., model A). The first layer can be defined as the layer that performs computation first when input data (Xn) 711 is input to the artificial intelligence model (e.g., model A). In trusted execution environment 320, processor 250 can compute the data input to the first layer based on the weights, biases, and / or activation functions of the first layer to prevent the input data (Xn) 711 from being exposed to the outside. However, this disclosure is not limited to the above description, and noise addition module 241 can also select a portion (n) of a layer from a plurality of layers that include the first layer. Noise addition module 241 can select a portion (n) of a layer based on specified information. In one embodiment, the specified information may include information indicating which layer(n) will be selected by the manufacturer of the artificial intelligence model (e.g., Model A) or an application associated with the artificial intelligence model (e.g., Model A) (e.g., Application A) so that the pre-computation operation will be performed. The noise addition module 241 may select a subset (n) of multiple layers included in the artificial intelligence model (e.g., Model A) based on the information indicating the layers. In another embodiment, the specified information may include information about the characteristics of the layers. Information about the characteristics of the layers may indicate whether the weight values ​​(or weight matrix (An)) included in the layer are linear. The noise addition module 241 may select some layers (n) with linear weights from multiple layers based on the characteristics of the layers. In another embodiment, the specified information may include information indicating the availability of idle resources in the electronic device 101. For example, the noise addition module 241 may determine the number of layers to be selected from multiple layers in proportion to the size of the idle resources based on the information indicating the availability of idle resources. In this case, the layers selected according to the determined number may be layers configured to have a higher priority than unselected layers. The priority may be predetermined by the manufacturer of the artificial intelligence model (e.g., Model A) or an application associated with the artificial intelligence model (e.g., Model A) (e.g., Application A). All of the above embodiments can be combined and executed, but this disclosure is not limited thereto, and only some embodiments may be executed.

[0079] According to various embodiments, the noise addition module 241 can generate noise values ​​(the aforementioned weight noise (zn) 631) for the weight values ​​(e.g., w) (or weight matrix (An)) to be applied to the selected layer (n). The value of the weight noise (zn) 631 can be generated using random values ​​within a specified range (e.g., a range below 0.9 or a range above 1.1), and the value of the weight noise (zn) 631 can be determined according to... Figure 2The computational capability of the computing device 220 described in Figure 4 is used to determine this, therefore, redundant descriptions will be omitted. The value of the weight noise (zn) 631 can be generated in different ranges specified for multiple layers, but this disclosure is not limited thereto, and the value of the weight noise (zn) 631 can be generated with the same value. The noise adding module 241 can apply the generated weight noise (zn) 631 to the weight value (w) (or weight matrix (An)) of the selected layer (n) (for example, multiplying the generated weight noise (zn) 631 with the weight value (or weight matrix (An)) of the selected layer), and can store the weight value (w') (or weight matrix (An') 633) to which the weight noise (zn) has been applied. Figure 6a As shown, the noise addition module 241 can store the weight matrix (An') with applied weighted noise (zn) 631 in a specific region 313 of the memory 230 (the specific region 313 is allocated to the rich execution environment 310) to allow its use in the rich execution environment 310. In another embodiment, the noise addition module 241 can store the weight matrix (An) in a specific region 313 of the memory 230 to allow its use in the rich execution environment 310. Figure 3c In a specific region 313a of the memory 230a shown.

[0080] According to various embodiments, the noise addition module 241 can generate noise values ​​(the aforementioned input noise (Rn) 651) to be added to the input data (Xn) 711 input to the selected layer (n). For example, the noise addition module 241 can generate a noise vector having a size corresponding to the size (or number) of the input data (Xn) input to the selected layer (n) as the input noise (Rn) 651. The values ​​of the input noise (Rn) 651 (e.g., r1, r2, r3, or r4) can be generated within a specified range (e.g., a range of values ​​that the input data (Xn) can have), and each value (e.g., r1, r2, r3, or r4) can be randomly generated. The noise addition module 241 can store the generated input noise (Rn) 651 in a specific region 313 in the memory 230 (the specific region 313 is allocated to the rich execution environment 310) to allow the generated input noise (Rn) 651 to be used in the rich execution environment 310.

[0081] According to various embodiments, such as Figure 6bAs shown, electronic device 101 (e.g., processor 250) can pre-store values ​​generated for each layer (n) selected in the trusted execution environment 320 (hereinafter referred to as pre-computed values). For example, pre-computed values ​​may include weight noise (zn) 631 for each selected layer (n), a weight matrix (An') 633 with weight noise (zn) 631 applied, an input noise vector (Rn) 651, a bias vector (Bn), and / or a noise subtraction matrix (Tn). The noise subtraction matrix (Tn) can be obtained as shown in [Equation 2] below, and as will be described below, the noise subtraction matrix (Tn) can be a matrix used to perform subtraction from computational data (Yn') 731 computed in the rich execution environment 310. When Tn is computed, each value can be implemented (+, -) in reverse to the description, and in this case, Tn can be subtracted from Yn' as described below.

[0082] [Equation 2]

[0083] In [Equation 2], n represents the identifier (or order) of the selected layers, An represents the weight matrix of each selected layer, Rn represents the input noise of each selected layer, zn represents the weight noise of each selected layer, and Bn represents the bias vector of each selected layer.

[0084] like Figure 6b As shown, at least a portion of the values ​​can be stored in a second region 321 allocated to the trusted execution environment 320 and can be used in the trusted execution environment 320. At least a portion of the values ​​(e.g., a weight matrix (An') to which noise values ​​are applied) can be stored in a specific region 313 within the first region 311 (the specific region 313 is pre-allocated in the rich execution environment 310) and can be used in the rich execution environment 310. In this case, for security reasons, the bias vector (Bn) and / or the noise subtraction matrix (Tn) in the values ​​may not be stored in the specific region 313 within the first region 311 (the specific region 313 is allocated in the rich execution environment 310) and may be stored only in the second region 321 allocated to the trusted execution environment 320. This disclosure is not limited to the description and / or illustration above, and for example, to reduce data storage, the input noise (Rn) and bias (Bn) may not be stored in these values, and only the noise subtraction matrix (Tn) may be stored.

[0085] According to various embodiments, in operation 503, electronic device 101 can determine whether an event for executing an artificial intelligence model has occurred. For example, such as... Figure 7As shown, electronic device 101 can identify the occurrence of events that enable the execution of an artificial intelligence model (e.g., model A) associated with the executed application (e.g., application A) based on the execution of a specified application (e.g., application A). In an embodiment, electronic device 101 can execute a specified artificial intelligence model based on the application to perform a function. Figure 5b As shown in 510, the camera application 511 can be implemented to process the acquired images or videos and provide a mode 513 for acquiring information about the object (or subject). Therefore, as... Figure 5b As shown in 520, when a user selects mode 513 on the execution screen of camera application 511, electronic device 101 can recognize that an event for processing data (or content) has occurred. Electronic device 101 can identify from a plurality of artificial intelligence models 231 an AI model (e.g., Model A) configured to correspond to camera application 511 (or implemented to provide subject recognition functionality corresponding to mode 513). The identified AI model (e.g., Model A) can be implemented to output information about the subject (e.g., the probability of the subject's presence) in response to input image data. Electronic device 101 can provide acquired data (e.g., images 512 and 521) as input data for computation based on the execution of camera application 511. In another embodiment, electronic device 101 can also execute a pre-configured AI model associated with the executed application. In this case, information about the artificial intelligence model corresponding to each application is pre-stored in electronic device 101, and electronic device 101 (e.g., processor 250) can identify and / or execute the artificial intelligence model corresponding to the application being executed by referring to the stored information.

[0086] According to various embodiments, when it is determined that an event for executing an artificial intelligence model has occurred, electronic device 101 can, in operation 505, acquire computational data based on calculations of at least a portion of data input to multiple layers using weight values ​​with applied noise values, and can, in operation 507, acquire output data based on the acquired computational data and the applied noise values. Figure 7As shown, when some layers (n) of an artificial intelligence model that has already performed pre-computation operations perform computations based on the multiple layers of the model where execution events have already occurred, the electronic device 101 can apply input noise (Rn) 651 to the data (Xn) 711 input to the layer in the trusted execution environment 320 to obtain data (Xn') 713 with input noise (Rn) 651 applied (operation 701); the computed data (Yn') 731 can be obtained by calculating the data (Xn') 713 with input noise (Rn) 651 applied and the weight value (or weight matrix (An') 633) with weight noise (zn) 631 applied in the rich execution environment 310 (operation 703); and the noise value can be subtracted from the computed data (Yn') 731 in the trusted execution environment 320 (operation 705). Operations 701, 703, and 705 can be defined as layer computation operations. To execute operations 701, 703, and 705, the execution environment of electronic device 101 can be switched (or changed). Electronic device 101 (e.g., processor 250) can execute an artificial intelligence model (e.g., model A) by switching (changing or configuring) its execution environment to trusted execution environment 320 based on the occurrence of an event for executing the artificial intelligence model (e.g., model A). Subsequently, operation 701 is executed in trusted execution environment 320, and after operation 701 completes, the execution environment of electronic device 101 can be changed from trusted execution environment 320 to rich execution environment 310. Operation 703 is executed in rich execution environment 310, and after operation 703 completes, the execution environment of electronic device 101 can be changed from rich execution environment 310 to trusted execution environment 320, and operation 705 can be executed in trusted execution environment 320. Electronic device 101 (e.g., processor 250) can perform computations for the first layer of the artificial intelligence model (e.g., model A) in trusted execution environment 320. For example, for the first layer of an artificial intelligence model (e.g., model A), electronic device 101 can perform normal calculations based on weight values ​​(A1), biases (B1), and activation functions, and can continuously perform the aforementioned layer calculation operations (operations 701, 703, and 705) in at least a portion of the remaining layers by using data output as a result of the calculations. The layer calculation operations (operations 701, 703, and 705) will be further described below.

[0087] According to various embodiments, when the layer used to perform the current computation corresponds to the layer performing the pre-computation operation, the processor 250 (e.g., noise addition module 241) can obtain input data (Xn') 713 with input noise (Rn) 651 applied by applying (e.g., adding) input noise (Rn) 651 to the input data (Xn) 711 input to that layer in the trusted execution environment 320. Figure 6bThe pre-stored information described herein is used to identify input noise (Rn) 651 for the layer performing the current computation, and the identified input noise (Rn) 651 can be applied (e.g., added) to input data (Xn) 711. The noise addition module 241 can store the input data (Xn') 713 with the applied input noise (Rn) 651 in a specific region 313 of the memory 230, wherein the specific region 313 is allocated to the rich execution environment 310. When the input data (Xn') 713 with the applied input noise (Rn) 651 is stored, the processor 250 can switch the execution environment to the rich execution environment 310.

[0088] According to various embodiments, processor 250 (e.g., computation execution module 243) can acquire computed data (Yn') 731 in rich execution environment 310 by computing (e.g., performing matrix multiplication) input data (Xn') 713 with applied input noise (Rn) 651 and a weight matrix (An') 633 with applied weight noise (zn) 631. Computation execution module 243 can access a specific region 313 in memory 230 (the specific region 313 is allocated in rich execution environment 310) and can identify the input data (Xn') 713 with applied input noise (Rn) 651 and the weight matrix (An') 633 with applied weight noise (zn). Computation execution module 243 can store the computed data (Yn') 731 generated according to the computation in the specific region 313 in memory 230, wherein the specific region 313 is allocated in rich execution environment 310. When storing computational data (Yn') 731, processor 250 can switch the execution environment to trusted execution environment 320.

[0089] According to various embodiments, processor 250 (e.g., output data acquisition module 245) can acquire output data (Yn) 751 in trusted execution environment 320 by subtracting noise values ​​from computed data (Yn') 731. For example, as shown in [Equation 3] below, output data acquisition module 245 can acquire output data by subtracting pre-stored weighted noise (zn) 631 from computed data (Yn') 731 (e.g., dividing computed data (Yn') 731 by pre-stored weighted noise (zn) 631) and adding it to the noise subtraction matrix (Tn). When calculating Tn, each value (+, -) can be implemented in reverse to the description, and in this case, Tn can be subtracted from Yn' as described below. This disclosure is not limited to the above description, and An can be subtracted from computed data (Yn') 731 when there is no bias in the corresponding layer. Rn / zn, instead of adding it to the noise subtraction matrix.

[0090] [Equation 3]

[0091] In [Equation 3], n represents the identifier (or order) of the layer performing the computation, Yn represents the output data, Yn' represents the computation data, zn represents the weight noise of each layer, and Tn represents the noise subtraction matrix in [Equation 3].

[0092] According to various embodiments, the processor 250 can use the acquired output data (Yn) 751 as input data for the next layer to continuously perform calculations. In this case, when an activation function exists in the layer where the current calculation is being performed, the processor 250 can continue to perform calculations by using the data output by applying the activation function to the output data as input data for the next layer.

[0093] According to various embodiments, when the computation of the current layer is completed, the processor 250 can remove the pre-calculated values ​​of the current layer (e.g., weight noise (zn) 631, weight matrix (An') 633 with weight noise (zn) 631 applied, input noise vector (Rn) 651, bias vector (Bn) and / or noise subtraction matrix (Tn)) from the memory 230, but this disclosure is not limited thereto.

[0094] Various embodiments illustrate that when an event for executing an artificial intelligence model has occurred in operation 503, electronic device 101 performs calculations based on pre-calculated values ​​in operation 505. However, this disclosure is not limited, and operation 503 can be performed after the event for executing an artificial intelligence model has occurred. For example, when an event for executing an artificial intelligence model has occurred, electronic device 101 can generate values ​​for a layer (e.g., weight noise (zn) 631 for each layer in the selected layer (n), a weight matrix (An') 633 with applied weight noise (zn), an input noise vector (Rn) 651, a bias vector (Bn), and / or a noise subtraction matrix (Tn)), and perform calculations for that layer based on the generated values.

[0095] Hereinafter, examples of the operation of electronic device 101 according to various embodiments are described.

[0096] According to various embodiments, when the electronic device 101 is in an idle state, the electronic device 101 can perform pre-computation operations for the artificial intelligence model 231.

[0097] Figure 8 This is a flowchart 800 illustrating examples of the operation of an electronic device 101 according to various embodiments. Figure 8 The order of operations shown is not limited to the order shown, and can be performed in various orders. Furthermore, according to various embodiments, operations can be performed in a more... Figure 8 The operations shown are more operations or more Figure 8 The operations shown are at least one of fewer operations. In the following text, refer to... Figure 9 describe Figure 8 .

[0098] Figure 9 Examples of idle and active states of electronic device 101 according to various embodiments are shown.

[0099] According to various embodiments, in operation 801, electronic device 101 may determine whether the state of the electronic device corresponds to an idle state. For example, the state of electronic device 101 may include an idle state and an active state. The idle state indicates to the processor (e.g., Figure 3a and Figure 3b Processor 250 or Figure 3c The second processor 250b) has sufficient resources to perform operations in the trusted execution environment 320, and the activity status indicates the processor (e.g., Figure 3a and Figure 3b Processor 250 or Figure 3c The second processor 250b) does not have sufficient resources to perform operations within the trusted execution environment 320. Resources may include processors (e.g., Figure 3a And 3B processor 250 or Figure 3c The use of the second processor 250b) or by a processor (e.g., Figure 3a And 3B processor 250 or Figure 3c The second processor 250b executes at least one of the number of processes (or operations, programs, or applications). Processor usage can be monitored by means of various types of time counters. In the following, an example implementation of the above according to electronic device 101 will be described (e.g., Figure 3a Implementation examples Figure 3b Implementation examples and Figure 3c Examples of operations for determining the idle state of electronic device 101 in the implementation examples.

[0100] According to various embodiments, such as Figure 3a and Figure 3bAs shown, when electronic device 101 is implemented in a single processor 250 and processor 250 has sufficient resources, electronic device 101 can determine that its state corresponds to an idle state. For example, when the resources of processor 250 have a value equal to or greater than a pre-configured value, electronic device 101 can determine that its state corresponds to an idle state, and when the resources have a value less than the pre-configured value, electronic device 101 can determine that its state corresponds to an active state. In an embodiment, when the usage of processor 250 has a value equal to or greater than a pre-configured value, electronic device 101 determines that its state corresponds to an active state, and when the usage of processor 250 has a value less than the pre-configured value, electronic device 101 can determine that its state corresponds to an idle state. For example, as... Figure 9 As shown in 901, the use of processor 250, identified as processor 250 executing at least one application, can have a value equal to or greater than a pre-configured value. At least one application can execute in foreground or background mode. In this case, electronic device 101 can determine the state of electronic device 101 corresponding to the active state. In another example, when the number of applications executing is small (e.g., ... Figure 9 (as shown in 902), or when no application is running (such as...). Figure 9 As shown in 903, the use of processor 250 may have a value less than a preset value. In this case, electronic device 101 can determine that the state of electronic device 101 corresponds to an idle state. In another embodiment, when the number of applications (or processors or programs) executing is equal to or greater than a preset value, electronic device 101 can determine that the state of electronic device 101 corresponds to an active state, and when the number of applications (or processors or programs) executing is less than a preset value, electronic device 101 can determine that the state of electronic device 101 corresponds to an idle state.

[0101] According to various embodiments, when the electronic device 101 is implemented as follows Figure 3cWhen multiple processors are shown (e.g., first processor 250a and second processor 250b), electronic device 101 can determine its state corresponding to an idle state if sufficient resources of the processor (e.g., second processor 250b) are allocated to the trusted execution environment 320. As described above, electronic device 101 can determine its state (e.g., idle or active state) based on a comparison between a pre-configured value and the number of applications (or processes or programs) executed by the processor (e.g., second processor 250b) or the usage of the processor (e.g., second processor 250b) allocated to the trusted execution environment 320; therefore, redundant descriptions will be omitted. The amount of resources of the processor (e.g., second processor 250b) can vary depending on whether a program (e.g., security tool) is being executed only for the trusted execution environment 320 by the processor (e.g., second processor 250b) allocated to the trusted execution environment 320.

[0102] According to various embodiments, when the state of electronic device 101 corresponds to an idle state, electronic device 101 may perform at least one pre-computation operation in operation 803. For example, in trusted execution environment 320, electronic device 101 may, based on determining that the state of electronic device 101 is idle, perform at least one of the following operations for at least a portion of multiple artificial intelligence models 231: operation 601 (selecting at least a portion of layers (n) from multiple layers of artificial intelligence model 231), applying weight noise (zn) to the weight values ​​(e.g., w) (or weight matrix (An)) of the selected layer (n) to generate weight values ​​(e.g., w') (or weight matrix (An') with applied noise (zn)) 633 (operation 603), or operation 605 (generating input noise (Rn) 651 to be applied to input data (Xn) to be input in the selected layer (n) Each operation (operations 601, 603, and 605) can be performed as described above in operation 501 of electronic device 101, therefore, redundant descriptions will be omitted.

[0103] According to various embodiments, when the determined state of electronic device 101 corresponds to an active state, electronic device 101 may not perform pre-calculation operations.

[0104] According to various embodiments, electronic device 101 may determine in operation 805 whether the state of the electronic device corresponds to an active state, and when the state of the electronic device corresponds to an active state, electronic device 101 may suspend the pre-computation operation in operation 807. For example, during the execution of the pre-computation operation, electronic device 101 may periodically and / or non-periodically determine the state of electronic device 101. The operation of determining the state of electronic device 101 may be performed as described in operation 801 above, and therefore, redundant descriptions will be omitted. When it is determined that the state of electronic device 101 corresponds to an active state, electronic device 101 may suspend the pre-computation operation. Thereafter, when the state of electronic device 101 changes to an idle state and the pre-computation operation of artificial intelligence model 231 has not yet been completed, electronic device 101 may continue to perform the pre-computation operation. For example, for a layer of artificial intelligence model 231 where at least one of the weight noise application or input noise generation has not been performed, electronic device 101 may continue to perform the operation of applying weight noise or generating input noise.

[0105] According to various embodiments, when it is determined that the state of electronic device 101 corresponds to an idle state, electronic device 101 can continue to perform pre-calculation operations.

[0106] Hereinafter, examples of the operation of electronic device 101 according to various embodiments are described.

[0107] According to various embodiments, electronic device 101 can perform pre-computation operations for a specific artificial intelligence model 231 until, based on the execution of the pre-computation operations for the specific artificial intelligence model 231, weighted noise is applied and the number of layers generating input noise is equal to or greater than (or exceeds) a pre-configured number.

[0108] Figure 10 This is a flowchart 1000 illustrating examples of the operation of an electronic device 101 according to various embodiments. Figure 10 The order of operations shown is not limited to the order shown, and can be performed in various orders. Furthermore, according to various embodiments, operations can be performed in a more... Figure 10 The operations shown are more operations or more Figure 10 The operation shown is at least one of fewer operations. In the following text, reference will be made to... Figure 11 describe Figure 10 .

[0109] Figure 11 Examples of operations are shown for performing pre-computation operations on a selected artificial intelligence model in the idle state of electronic device 101 according to various embodiments.

[0110] According to various embodiments, in operation 1001, electronic device 101 may determine whether the state of electronic device 101 corresponds to an idle state, and when the state of electronic device 101 corresponds to an idle state, the electronic device may select one of a plurality of artificial intelligence models in operation 1003. For example, based on pre-configured values ​​and a processor (e.g., Figure 3a and Figure 3b Processor 250 or Figure 3c The electronic device 101 compares the resources of the second processor 250b) and determines that its state corresponds to an idle state when the resources have a value equal to or greater than a pre-configured value, and that its state corresponds to an active state when the resources have a value less than the pre-configured value. The operation of determining the state of the electronic device 101 can be performed as shown in the operation 801 of the electronic device 101 described above; therefore, redundant descriptions will be omitted. When the state of the electronic device 101 corresponds to an idle state, the electronic device 101 can select an artificial intelligence model 1100 from among the multiple artificial intelligence models 231 for which pre-computation operations will be performed. For example, the electronic device 101 can select at least one artificial intelligence model 1100 from among the multiple artificial intelligence models 231 that has not yet completed pre-computation. When pre-computation is completed, this may mean that the number of layers among the multiple layers included in the artificial intelligence model that have completed pre-computation operations is equal to or greater than the pre-configured value. In one embodiment, when selecting an AI model 1100, the electronic device 101 may randomly select AI model 1100 from the remaining AI models among a plurality of AI models 231 that have not yet completed pre-computation operations. In another embodiment, when selecting an AI model 1100, the electronic device 101 may select AI model 1100 from the remaining AI models among a plurality of AI models 231 that have not yet completed pre-computation operations according to a specified order. Selecting AI model 1100 according to a specified order may include selecting a higher-priority AI model earlier than an AI model 1100 with a lower priority. The priority may be configured proportionally to the complexity of the AI ​​model. For example, the more layers included in an AI model or the greater the capabilities of the AI ​​model, the higher the complexity of the AI ​​model can be determined. In another embodiment, when the manufacturer predetermines that a particular AI model does not require pre-computation, the electronic device 101 may not perform pre-computation operations for that particular AI model. In this case, each of the plurality of AI models 231 may include information indicating whether pre-computation operations are required, and the electronic device 101 may perform pre-computation operations only for AI models that include information indicating that pre-computation is required.

[0111] According to various embodiments, electronic device 101 may, in operation 1005, compare a pre-configured number with the number of layers for performing pre-computation operations on a selected artificial intelligence model; when the number of layers is less than the pre-configured number, select one of a plurality of layers in operation 1007, and generate a pre-computation value for the selected layer in operation 1009. For example, as Figure 11 As shown in 1101, electronic device 101 can identify whether pre-computation operations have been performed for multiple layers (e.g., first layer, second layer, third layer... and nth layer) included in the selected artificial intelligence model 1100. In an embodiment, electronic device 101 can identify pre-computation values ​​for at least a portion of the multiple layers of the selected artificial intelligence model 1100 stored in memory (e.g., a specific region 313a in a second region 321 allocated to a trusted execution environment 320 or a first region 311 allocated to a rich execution environment 310) to determine whether pre-computation operations have been performed for that layer. As described above... Figure 6b As described, the pre-computed values ​​may include weight noise (zn) for each layer (n), a weight matrix (An') with applied weight noise (zn), an input noise vector (Rn), a bias vector (Bn), and / or a noise subtraction matrix (Tn). Electronic device 101 can determine that, among multiple layers, the layer storing the pre-computed values ​​corresponds to the layer that has completed the pre-computation operation, and the layer not storing the pre-computed values ​​corresponds to the layer that has not completed the pre-computation operation. In an embodiment, electronic device 101 can identify information stored in memory (e.g., a specific region 313a in a second region 321 allocated to a trusted execution environment 320 or a first region 311 allocated to a rich execution environment 310) indicating the completion of pre-computation for a layer of the selected artificial intelligence model 1100 in order to determine whether a pre-computation operation has been completed for that layer. For example, after completing the pre-computation operation for a specific layer, the electronic device 101 can store information indicating the completion of the pre-computation operation (e.g., identification information of the specific layer that has completed the pre-computation operation) in memory (e.g., a specific region 313a in a second region 321 allocated to a trusted execution environment 320 or a first region 311 allocated to a rich execution environment 310). The electronic device 101 can identify the information indicating the completion of the pre-computation operation (which is pre-stored in memory) to determine that the layer including the information corresponds to the layer that has completed the pre-computation operation, and the layer not including the information corresponds to the layer that has not yet completed the pre-computation operation.

[0112] According to various embodiments, electronic device 101 can identify the number of layers that have completed pre-computation operations based on determining whether pre-computation operations have been completed for a layer. In embodiments, after performing pre-computation operations on a selected artificial intelligence model 1100, electronic device 101 can pre-store information about the number of layers that have completed pre-computation operations, and can identify the number of layers that have completed pre-computation operations for the selected intelligent model 1100 based on the identification of the stored information. Electronic device 101 can compare a pre-configured value (ε) with the number of layers that have completed pre-computation operations, and based on the comparison result, when the number of layers that have completed pre-computation operations is less than the pre-configured value (ε), electronic device 101 can perform pre-computation operations on at least a portion of the layers that have not yet completed pre-computation operations, such as... Figure 11 As shown in 1102. As described above, the pre-computation operation may include: applying weight noise (zn) to the weight values ​​(e.g., w) (or weight matrix (An)) of the selected layer (n) and generating weight values ​​with applied noise (e.g., w') (or weight matrix (An') with applied weight noise (zn) 633) (operation 603), and generating input noise (Rn) 651 to be applied to the input data (Xn) to be input to the selected layer (n) (operation 605). Therefore, redundant descriptions will be omitted. Based on the comparison between the number of layers for which the pre-computation operation has been completed and the pre-configuration value (ε), when the number of layers for which the pre-computation operation has been completed is greater than the pre-configuration value (ε), the electronic device 101 may select another artificial intelligence model, wherein the number of layers for which the pre-computation operation has been completed for the other artificial intelligence model is less than the pre-configuration value (ε). The pre-configuration value (ε) may be a pre-stored value associated with multiple artificial intelligence models 231, but may be adjusted. In addition, different values ​​(ε) may be configured for each of the multiple artificial intelligence models 231. For example, the value (ε) can be pre-configured by the manufacturer of each of the multiple artificial intelligence models 231.

[0113] According to various embodiments, electronic device 101 may randomly select a layer from layers from which pre-computation operations have not yet been completed, and perform the pre-computation operations. In this case, as described above, electronic device 101 may not perform pre-computation operations for the first layer (e.g., the first layer). In embodiments, electronic device 101 may select layers based on their priority among the layers from which pre-computation operations have not yet been completed, and perform the pre-computation operations. For example, the priority may be predetermined by the manufacturer of artificial intelligence model 1100 or the application associated with artificial intelligence model 1100. In another example, the priority of each layer of artificial intelligence model 1100 may be determined to be higher when the capacity of the values ​​(e.g., weight matrix) included in that layer is larger.

[0114] According to various embodiments, after operations 1007 and 1009, electronic device 101 may compare the pre-configured quantity with the number of layers that have completed pre-computation operations in operation 1011, and may re-execute operations 1007 and 1009 if the number of layers that have completed pre-computation operations is less than the pre-configured quantity. For example, electronic device 101 may select at least one layer from layers that have not yet completed pre-computation operations, and after completing the pre-computation operations, perform an operation to compare the pre-configured value (ε) with the number of layers that have completed pre-computation operations again. Based on the comparison result, when the number of layers that have completed pre-computation operations is less than the pre-configured value (ε), electronic device 101 may perform pre-computation operations on at least a portion of the layers that have not yet completed pre-computation operations. Alternatively, when the number of layers that have completed pre-computation operations is greater than the pre-configured value (ε), electronic device 101 may complete (or pause) the pre-computation operations of the selected artificial intelligence model 1100.

[0115] Hereinafter, examples of the operation of electronic device 101 according to various embodiments are described.

[0116] According to various embodiments, when performing calculations on a specific artificial intelligence model, electronic device 101 can perform normal calculation operations on a layer of the specific artificial intelligence model (wherein, for that layer, pre-calculation operations have not yet been completed (or pre-calculation operations have not yet been performed)), and electronic device 101 can perform calculation operations on a layer for which pre-calculation operations have been performed based on pre-calculated values.

[0117] Figure 12 This is a flowchart 1200 illustrating examples of the operation of an electronic device 101 according to various embodiments. Figure 12 The order of operations shown is not limited to the order shown, and operations can be performed in various orders. Furthermore, according to various embodiments, operations can be performed in a more... Figure 10 The operations shown are more operations or more Figure 10 The operation shown is at least one of fewer operations. In the following text, reference will be made to... Figure 13a and 13b describe Figure 12 .

[0118] Figure 13a Examples of operations for performing computations on layers of an artificial intelligence model for electronic device 101 according to various embodiments are shown, wherein no pre-computation operation has been completed (or no pre-computation operation has been performed) for that layer. Figure 13b Examples of operations for performing calculations on layers of an artificial intelligence model for electronic device 101 according to various embodiments are shown, wherein pre-computation operations have been performed for the layer.

[0119] According to various embodiments, in operation 1201, electronic device 101 can identify the occurrence of an event for executing an artificial intelligence model and initiate computation based on the artificial intelligence model, and can select one of multiple layers of the artificial intelligence model in operation 1203. For example, electronic device 101 can obtain an event for executing an artificial intelligence model (e.g., artificial intelligence model A 1300) based on the execution of an application (e.g., inputting a model for processing data based on artificial intelligence model 1300). Based on the occurrence of the event, electronic device 101 can change the execution environment (or execution mode) to a trusted execution environment 320 and identify the artificial intelligence model 1300 on the trusted execution environment 320. Electronic device 101 can sequentially select layers from multiple layers included in the identified artificial intelligence model 1300. For example, electronic device 101 can select the first layer (e.g., the first layer) from multiple layers (e.g., the first layer, the second layer, the third layer, ..., the nth layer), such as Figure 13a As shown in the image.

[0120] According to various embodiments, in operation 1205, electronic device 101 can determine whether pre-computed values ​​for the selected layer exist. In embodiments, electronic device 101 can identify whether pre-computed values ​​(e.g., weight noise (zn), weight matrix (An') with applied weight noise (zn), input noise vector (Rn), bias vector (Bn), and / or noise subtraction matrix (Tn)) of the selected layer (e.g., the first layer or the second layer) of the artificial intelligence model 1300 exist in memory (e.g., a second region 321 allocated to the trusted execution environment 320 or a specific region 313a in the first region 311 allocated to the rich execution environment 310) (or stored in electronic device 101). In an embodiment, the electronic device 101 may determine whether a pre-calculated value exists for the currently selected layer based on the identification of information indicating whether pre-calculation for the selected layer (e.g., the information is pre-stored in memory) indicating whether pre-calculation for the selected layer (e.g., the existence of a pre-calculated value is determined when there is information indicating that pre-calculation for the layer has been completed, or the absence of a pre-calculated value is determined when there is no information indicating that pre-calculation for the layer has been completed).

[0121] According to various embodiments, when no pre-calculated value exists for the selected layer, electronic device 101 can perform calculations based on the parameters of the current layer in a trusted execution environment during operation 1207. For example, electronic device 101 can determine that no pre-calculated value exists for the currently selected layer (e.g., the first layer), such as... Figure 13aAs shown in the diagram. In this case, electronic device 101 can perform computations on input data in trusted execution environment 320 based on at least one parameter of the selected layer (layer #n) (e.g., the first layer) (e.g., at least one of the weight matrix (An) (e.g., A1), activation function, or bias). For example, electronic device 101 (e.g., processor 250) can acquire input data (Xn) and perform computations (e.g., matrix multiplication) on the weight values ​​(e.g., weight matrix (An) (e.g., A1)) of the currently selected layer (e.g., the first layer) and the acquired input data (Xn) in trusted execution environment 320. Electronic device 101 can perform computations (e.g., matrix addition) on the computation result and the bias (Bn) of the layer (e.g., the first layer) to obtain output data output by applying an activation function when an activation function (e.g., ReLU) is present. Computational operations in layers of an artificial intelligence model can be performed using methods known in the art, and therefore their detailed description will be omitted. After acquiring the output data, the electronic device 101 determines in operation 1217 whether the current layer corresponds to the last layer, and when the current layer does not correspond to the last layer, it can continue to sequentially select the next layer (e.g., the second layer) and perform calculations in operation 1203.

[0122] Optionally, this disclosure is not limited to the above description and / or explanation, and when there is no pre-calculated value for the selected layer, the electronic device 101 may also perform calculation operations on the currently selected layer based on the pre-calculated value of another layer.

[0123] According to various embodiments, when a pre-calculated value for the selected layer exists, electronic device 101 can acquire the pre-calculated value for the current layer in operation 1209, and can add input noise to the input data based on the acquired pre-calculated value in operation 1211 to generate first input data. For example, when such... Figure 13b When it is determined that a pre-calculated value exists for the currently selected layer (layer #n+1) (e.g., the second layer), electronic device 101 can perform calculations based on the pre-calculated value. For example, electronic device 101 can add input noise to the data (e.g., the output data of the first layer) (Xn+1) input to the current layer (layer #n+1) (e.g., the second layer) in the trusted execution environment 320 to obtain input data (Xn+1') with applied noise, and can store the input data (Xn+1') with applied noise in a specific region 313a in the first region 311 allocated to the rich execution environment 310. Operation 1211 of electronic device 101 can be performed in the same manner as the above-described operation (operation 505) of electronic device 101 obtaining calculated data, therefore, redundant descriptions will be omitted.

[0124] According to various embodiments, in operation 1213, electronic device 101 can perform calculations on the generated first input data and weight values ​​with applied weight noise to obtain first calculated data. For example, electronic device 101 can switch (or change) the execution environment from a trusted execution environment 320 to a rich execution environment 310, and in the rich execution environment 310, obtain input data (Xn+1') stored in a specific region 313a in the first region 311 and having noise values ​​applied thereto, as well as weight values ​​(An+1') of the current layer (e.g., the second layer), wherein the weight values ​​have weight noise applied thereto. Electronic device 101 can perform calculations (e.g., matrix multiplication) on the input data (Xn+1') and the weight values ​​(An+1') of the current layer (e.g., the second layer) to obtain calculated data (Yn+1'), wherein the weight values ​​have weight noise applied thereto. Electronic device 101 can store the obtained calculated data (Yn+1') in a specific region 313a in the first region 311. Operation 1213 of electronic device 101 can be performed in the same manner as operation 505 of electronic device 101 for acquiring computational data; therefore, redundant descriptions will be omitted.

[0125] According to various embodiments, in operation 1215, electronic device 101 can acquire first output data based on first computational data and pre-stored noise values. For example, electronic device 101 can switch the execution environment from rich execution environment 310 to trusted execution environment 320 and acquire computational data (Yn+1') of the current layer (e.g., the second layer) from a specific region 313a in the first region 311. Electronic device 101 can remove noise from the computational data (Yn+1') (e.g., divide the computational data (Yn+1') by the input noise (zn+1) and add it to Tn+1) to acquire the output data (Yn+1). Operation 1213 of electronic device 101 can be performed in the same manner as the operation of acquiring output data (operation 507) of electronic device 101, therefore, redundant descriptions will be omitted.

[0126] According to various embodiments, electronic device 101 can determine in operation 1217 whether the current layer corresponds to the last layer, and can obtain result data from the output layer in 1219 when the current layer corresponds to the last layer. For example, when the currently selected layer corresponds to the last layer, electronic device 101 can obtain the output data of the last layer as result data.

[0127] According to various embodiments, when the current layer does not correspond to the last layer, in operation 1203, the electronic device 101 can select one of a plurality of layers of the artificial intelligence model to perform calculations on the selected layer. For example, the electronic device 101 can determine whether the current layer corresponds to the last layer, and can continue to sequentially select the next layer (e.g., the third layer) to perform calculations when the current layer does not correspond to the last layer.

[0128] According to various embodiments, an electronic device (e.g., Figure 2 Electronic device 101), the electronic device (e.g., Figure 2 Electronic device 101) includes memory (e.g., Figure 2 The memory 230) and at least one processor (e.g., Figure 3a and Figure 3b Processor 250, Figure 3c The first processor 250a and the second processor 250b), wherein at least one processor (e.g., Figure 3a and Figure 3b Processor 250, Figure 3c The first processor 250a and the second processor 250b are configured to: pass noise values ​​(e.g., Figure 6a zn) is used in storage in electronic devices (e.g., Figure 2 The weight values ​​of at least a portion of the multiple layers included in the artificial intelligence model of the electronic device 101 (e.g., Figure 6a An), to obtain the applied noise value (e.g., Figure 6a The weight values ​​of zn) (e.g., Figure 6a When an event is identified that is used to execute an artificial intelligence model, noise values ​​(e.g., An') are applied. Figure 6a The weight values ​​of zn) (e.g., Figure 6a An'), based on calculations of at least a portion of the data input to the plurality of layers, obtain computational data (e.g., Figure 7 Yn'); and based on the obtained computational data (e.g., Figure 7 Yn') and the applied noise value (e.g., Figure 6a (zn) to obtain output data (e.g., Figure 7 Yn).

[0129] According to various embodiments, an electronic device (e.g., electronic device 101) may be provided, wherein at least one processor (e.g., Figure 3a and Figure 3b Processor 250, Figure 3c The first processor 250a and the second processor 250b are configured to perform operations based on multiple execution environments, including rich execution environments (e.g., Figures 3a to 3c Rich execution environment 310) and trusted execution environment (e.g., Figures 3a to 3c Trusted Execution Environment 320), and at least one processor (e.g., Figure 3a and Figure 3b Processor 250, Figure 3c The first processor 250a and the second processor 250b are configured to: in a trusted execution environment (e.g., Figures 3a to 3c In a trusted execution environment (320), noise values ​​are generated (e.g., Figure 6a zn) and noise values ​​(e.g., Figure 6a The zn) is applied to the weight values ​​of at least a portion of multiple layers (e.g., Figure 6a (An); when an event is detected, in a rich execution environment (e.g., Figures 3a to 3c In the rich execution environment 310), by using the applied noise value (e.g., Figure 6a The weight values ​​of zn) (e.g., Figure 6a An'), computes on at least a portion of the data input to the plurality of layers; and in a trusted execution environment (e.g., Figures 3a to 3c In a trusted execution environment 320, based on the obtained computational data (e.g., Figure 7 Yn') and the applied noise value (e.g., Figure 6a (zn) to obtain output data (e.g., Figure 7 Yn).

[0130] According to various embodiments, electronic devices (e.g., Figure 2 Electronic device 101), wherein the first part of the memory (e.g., Figure 2 The memory 230) is allocated to rich execution environments (e.g., Figure 3a and Figure 3c The rich execution environment 310), the second part of the memory (e.g., Figure 2 The memory 230) is allocated to a trusted execution environment (e.g., Figures 3a to 3c Trusted Execution Environment 320), and at least one processor (e.g., Figure 3a and Figure 3b Processor 250, Figure 3c The first processor 250a and the second processor 250b are configured to: in a trusted execution environment (e.g., Figures 3a to 3c In the Trusted Execution Environment 320, noise values ​​(e.g., Figure 6a The weight values ​​of zn) (e.g., Figure 6a An') is stored in the first part of the memory (e.g., Figure 2 In the memory 230), and in a rich execution environment (e.g., Figures 3a to 3cIn the rich execution environment 310), the applied noise value was obtained (e.g., Figure 6a The weight values ​​of zn) (e.g., Figure 6a (An').

[0131] According to various embodiments, electronic devices (e.g., Figure 2 Electronic device 101), wherein at least one processor (e.g., Figure 3a and Figure 3b Processor 250, Figure 3c The first processor 250a and the second processor 250b are configured to select noise values ​​from a specified range of values ​​(e.g., Figure 6a zn), and the specified range of values ​​includes values ​​less than 0.9 or values ​​equal to or greater than 1.1.

[0132] According to various embodiments, electronic devices (e.g., Figure 2 Electronic device 101), wherein the electronic device (e.g., Figure 2 The electronic device 101 may further include a plurality of computing devices for performing computations based on an artificial intelligence model, wherein the plurality of computing devices are allocated to rich execution environments (e.g., in multiple execution environments) within a plurality of execution environments. Figures 3a to 3c A rich execution environment 310), and at least one processor (e.g., Figure 3a and Figure 3b Processor 250, Figure 3c The first processor 250a and the second processor 250b are configured to operate in a rich execution environment (e.g., Figures 3a to 3c In the rich execution environment 310), the plurality of computing devices are used based on the applied noise value (e.g., Figure 6a The weight values ​​of zn) (e.g., Figure 6a An') performs calculations on at least a portion of the data input to multiple layers.

[0133] According to various embodiments, electronic devices (e.g., Figure 2 Electronic device 101), wherein at least one processor (e.g., Figure 3a and Figure 3b Processor 250, Figure 3c The first processor 250a and the second processor 250b are configured to operate in a trusted execution environment (e.g., Figures 3a to 3c In a trusted execution environment 320, a first noise value (e.g.,) is generated corresponding to at least a portion of the input data to be input into multiple layers. Figure 6a (Rn).

[0134] According to various embodiments, electronic devices (e.g., Figure 2Electronic device 101), wherein at least one processor (e.g., Figure 3a and Figure 3b Processor 250, Figure 3c The first processor 250a and the second processor 250b are configured to generate a first noise value (e.g., by randomly selecting values ​​from a range of values ​​of the input data to be input) Figure 6a Each of Rn).

[0135] According to various embodiments, electronic devices (e.g., Figure 2 Electronic device 101), wherein at least one processor (e.g., Figure 3a and Figure 3b Processor 250, Figure 3c The first processor 250a and the second processor 250b are configured to: when an event for executing an artificial intelligence model is identified, in a trusted execution environment (e.g., Figures 3a to 3c In the trusted execution environment 320), by using the first noise value (e.g., Figure 6a Rn) is applied to at least a portion of the data input to multiple layers to obtain a first noise value (e.g., Figure 6a The data of Rn), and based on the obtained application of the first noise value (e.g., Figure 6a The data of Rn) will transfer the execution environment of the electronic device (e.g., electronic device 101) from a trusted execution environment (e.g., Figures 3a to 3c The trusted execution environment (320) is changed to a rich execution environment (e.g., Figures 3a to 3c The rich execution environment 310).

[0136] According to various embodiments, electronic devices (e.g., Figure 2 Electronic device 101), wherein at least one processor (e.g., Figure 3a and Figure 3b Processor 250, Figure 3c The first processor 250a and the second processor 250b are configured to operate in a rich execution environment (e.g., Figures 3a to 3c In the rich execution environment 310), based on the application of noise values ​​(e.g., Figure 6a The weight values ​​of zn) (e.g., Figure 6a An') was applied to the first noise value (e.g., Figure 6a The data of Rn are used for calculation.

[0137] According to various embodiments, electronic devices (e.g., Figure 2 Electronic device 101), wherein at least one processor (e.g., Figure 3a and Figure 3b Processor 250, Figure 3cThe first processor 250a and the second processor 250b are configured to: in a trusted execution environment (e.g., Figures 3a to 3c In a trusted execution environment 320, the weight values ​​of at least a portion of multiple layers based on an artificial intelligence model (e.g., Figure 6a An), noise value (e.g., Figure 6a zn), first noise value (e.g., Figure 6a The first value is generated by using Rn and the bias, and the generated first value is stored in memory (e.g., Rn). Figure 2 In a part of the memory 230.

[0138] According to various embodiments, electronic devices (e.g., Figure 2 Electronic device 101), wherein at least one processor (e.g., Figure 3a and Figure 3b Processor 250, Figure 3c The first processor 250a and the second processor 250b are configured to base their decisions on a pre-stored first value and in a rich execution environment (e.g., Figures 3a to 3c Computational data obtained in the rich execution environment 310 (e.g., Figure 7 The calculation of Yn' is used to obtain the output data (e.g., Figure 7 Yn).

[0139] According to various embodiments, electronic devices (e.g., Figure 2 Electronic device 101), wherein at least one processor (e.g., Figure 3a and Figure 3b Processor 250, Figure 3c The first processor 250a and the second processor 250b are configured to: identify electronic devices (e.g., Figure 2 The state of the electronic device 101), and when the electronic device (e.g., Figure 2 When the state of the electronic device 101 corresponds to the idle state, in a trusted execution environment (e.g., Figures 3a to 3c Noise values ​​are generated in a trusted execution environment 320 (e.g., Figure 6a zn).

[0140] According to various embodiments, electronic devices (e.g., Figure 2 Electronic device 101), wherein at least one processor (e.g., Figure 3a and Figure 3b Processor 250, Figure 3c The first processor 250a and the second processor 250b are configured to: identify noise values ​​generated for multiple layers of the artificial intelligence model (e.g., Figure 6aThe number of layers (zn); when the number of identified layers is less than a preset value, no noise value is ever generated for it (e.g., Figure 6a Select a subset of layers from the zn layer; and generate noise values ​​associated with the selected subset of layers (e.g., Figure 6a zn).

[0141] According to various embodiments, electronic devices (e.g., Figure 2 Electronic device 101), wherein at least one processor (e.g., Figure 3a and Figure 3b Processor 250, Figure 3c The first processor 250a and the second processor 250b are configured to suppress the generation of noise values ​​associated with the first layer of a plurality of layers in an artificial intelligence model (e.g., Figure 6a zn).

[0142] According to various embodiments, electronic devices (e.g., Figure 2 Electronic device 101), wherein at least one processor (e.g., Figure 3a and Figure 3b Processor 250, Figure 3c The first processor 250a and the second processor 250b are configured to: when an event is identified, identify noise values ​​(e.g., for a portion of multiple layers of the artificial intelligence model) Figure 6a Whether zn is stored in an electronic device (e.g., Figure 2 In the electronic device 101); when the electronic device (e.g., Figure 2 The electronic device 101 does not store noise values ​​(e.g., Figure 6a When zn), in a trusted execution environment (e.g., Figures 3a to 3c In a trusted execution environment 320, the weight values ​​(e.g., based on a portion of the plurality of layers) are used. Figure 6a An) calculates the data input to the plurality of layers; and when the noise value (e.g., Figure 6a zn) is stored in an electronic device (e.g., Figure 2 When in an electronic device 101, in a rich execution environment (e.g., Figures 3a to 3c In the rich execution environment 310), noise values ​​(e.g., based on a portion of the plurality of layers) are applied. Figure 6a The weight values ​​of zn) (e.g., Figure 6a An'), calculates the data input to the plurality of layers.

[0143] According to various embodiments, an electronic device (e.g., Figure 2 A method of operating an electronic device 101, the method comprising: by adjusting the noise value (e.g., Figure 6a zn) is used in storage in electronic devices (e.g., Figure 2 The weight values ​​of at least a portion of the multiple layers included in the artificial intelligence model of the electronic device 101 (e.g., Figure 6a An), to obtain the applied noise value (e.g., Figure 6a The weight values ​​of zn) (e.g., Figure 6a An'); when an event is identified for executing an artificial intelligence model, noise values ​​(e.g., are applied) are used. Figure 6a The weight values ​​of zn) (e.g., Figure 6a An'), based on computations of at least a portion of the data input to multiple layers, yields computational data (e.g., Figure 7 Yn'); and based on the acquired calculation data (e.g., Figure 7 Yn') and the applied noise value (e.g., Figure 6a (zn) to obtain output data (e.g., Figure 6a Yn).

[0144] According to various embodiments, a method of operation may be provided, wherein at least one processor (e.g., Figure 3a and Figure 3b Processor 250, Figure 3c The first processor 250a and the second processor 250b are configured to perform operations based on multiple execution environments, and the multiple execution environments include rich execution environments (e.g., Figures 3a to 3c Rich execution environment 310) and trusted execution environment (e.g., Figures 3a to 3c The trusted execution environment 320), and the operation method may further include: in the trusted execution environment (e.g., Figures 3a to 3c In a trusted execution environment (320), noise values ​​are generated (e.g., Figure 6a zn) and noise values ​​(e.g., Figure 6a The zn) is applied to at least a portion of the weight values ​​of multiple layers (e.g., Figure 6a (An); when an event is identified, in a rich execution environment (e.g., Figures 3a to 3c In the rich execution environment 310), by using the applied noise value (e.g., Figure 6a The weight values ​​of zn) (e.g., Figure 6a An') performs computation on at least a portion of the data input to multiple layers; and in a trusted execution environment (e.g., Figures 3a to 3c In a trusted execution environment 320, based on the obtained computational data (e.g., Figure 7 Yn') and the applied noise value (e.g., Figure 6a (zn) to obtain output data (e.g., Figure 7 Yn).

[0145] According to various embodiments, an operation method may be provided, in which a first portion of a memory (e.g., Figure 2 of the memory 230) is allocated to a rich execution environment (e.g., Figures 3a to 3c of the rich execution environment 310), and a second portion of the memory (e.g., Figure 2 of the memory 230) is allocated to a trusted execution environment (e.g., Figures 3a to 3c of the trusted execution environment 320), and the method may comprise: in the trusted execution environment (e.g., Figures 3a to 3c of the trusted environment 320), storing a weight value applied with a noise value (e.g., Figure 2 e.g., in the first portion of the memory (e.g., Figure 6a of zn) (e.g., Figure 6a of An'), wherein the first portion is allocated to the rich execution environment (e.g., Figures 3a to 3c of the rich execution environment 310); and obtaining, in the rich execution environment (e.g., Figures 3a to 3c of the rich execution environment 310), the weight value applied with the noise value (e.g., Figure 6a of zn) (e.g., Figure 6a of An').

[0146] According to various embodiments, an operation method may be provided, wherein the noise value (e.g., Figure 6a of zn) is configured to be selected from values within a specified range, and the values within the specified range include a value less than 0.9 or a value equal to or greater than 1.1.

[0147] According to various embodiments, an electronic device (e.g., Figure 2 of the electronic device 101) may be provided, and the electronic device (e.g., Figure 2 of the electronic device 101) comprises a memory (e.g., Figure 2 of the memory 230) and at least one processor (e.g., Figure 3a and Figure 3b of the processor 250, Figure 3c of the first processor 250a and the second processor 250b), wherein the at least one processor (e.g., Figure 3a and Figure 3b of the processor 250, Figure 3c of the first processor 250a and the second processor 250b) is configured to: in the trusted execution environment (e.g., Figures 3a to 3c of the trusted execution environment 320), by applying the noise value (e.g., Figure 6a of zn) to the weight values of at least a part of a plurality of layers included in an artificial intelligence model stored in the electronic device (e.g., Figure 2 of the electronic device 101) (e.g., Figure 6a An), to obtain the applied noise value (e.g., Figure 6a The weight values ​​of zn) (e.g., Figure 6a An'); when an event is identified that is used to execute an artificial intelligence model, in a rich execution environment (e.g., Figures 3a to 3c In the rich execution environment 310), by using the applied noise value (e.g., Figure 6a The weight values ​​of zn) (e.g., Figure 6a An'), based on computations of at least a portion of the data input to multiple layers, yields computational data (e.g., Figure 7 Yn'); based on the obtained calculation data (e.g., Figure 7 Yn'), will electronic devices (e.g., Figure 2 The state of the electronic device 101) is determined from the rich execution environment (e.g., Figures 3a to 3c The rich execution environment 310) is changed to a trusted execution environment (e.g., Figures 3a to 3c Trusted execution environment 320); and in a trusted execution environment (e.g., Figures 3a to 3c In a trusted execution environment 320, based on the obtained computational data (e.g., Figure 7 Yn') and the applied noise value (e.g., Figure 6a (zn) to obtain output data (e.g., Figure 7 Yn).

Claims

1. An electronic device comprising: Memory stores instructions; as well as At least one processor, wherein the at least one processor is configured to perform operations based on a plurality of execution environments, wherein the plurality of execution environments includes rich execution environments and trusted execution environments. When the instruction is executed by the at least one processor, the electronic device causes the electronic device to generate noise values ​​in the trusted execution environment. By multiplying the generated noise value with the weight values ​​of at least a portion of the multiple layers included in the artificial intelligence model stored in the electronic device in the trusted execution environment, the noise value is applied to the weight value to obtain a weight value with the noise value applied. When an event for executing the artificial intelligence model is identified, in the rich execution environment, computational data is obtained based on the computation of at least a portion of the data input to the plurality of layers using weights applied with the noise value; and In the trusted execution environment, output data is obtained based on the acquired computational data and the noise value applied to the weight values. Specifically, a first portion of the memory is allocated to the rich execution environment, and a second portion of the memory is allocated to the trusted execution environment. The artificial intelligence model is stored in a second portion of the memory, and this second portion is inaccessible to the at least one processor in the rich execution environment. The first portion of the memory is accessible to the at least one processor in both the rich execution environment and the trusted execution environment.

2. The electronic device according to claim 1, in, When the instruction is executed by the at least one processor, the electronic device: In the trusted execution environment, the weight values ​​applied to the noise values ​​are stored in the first part of the memory; as well as In the rich execution environment, the weight value applied to the noise value is obtained by accessing a first portion of the memory.

3. The electronic device according to claim 2, wherein, When the instruction is executed by the at least one processor, the electronic device: The noise value is selected from a specified range of values, and the specified range of values ​​includes values ​​less than 0.9 or values ​​equal to or greater than 1.

1.

4. The electronic device according to claim 1, further comprising: Multiple computing devices are used to perform computations based on the artificial intelligence model, wherein the multiple computing devices are allocated to the rich execution environment among the multiple execution environments. Wherein, when the instruction is executed by the at least one processor, the electronic device: In the rich execution environment, the data input to the multiple layers is computed based on weights applied to the noise values ​​using the multiple computing devices.

5. The electronic device according to claim 1, wherein, When the instruction is executed by the at least one processor, the electronic device: In the trusted execution environment, a first noise value is generated corresponding to the input data that will be input to at least a portion of the plurality of layers.

6. The electronic device according to claim 5, wherein, When the instruction is executed by the at least one processor, the electronic device: Each of the first noise values ​​is generated by randomly selecting values ​​from the range of values ​​in the input data.

7. The electronic device according to claim 5, wherein, When the instruction is executed by the at least one processor, the electronic device: When the event for executing the artificial intelligence model is identified, data with the applied first noise value is obtained by applying a first noise value to at least a portion of the data input to the plurality of layers in the trusted execution environment, and Based on the data obtained with the first noise value applied, the execution environment of the electronic device is changed from the trusted execution environment to the rich execution environment.

8. The electronic device according to claim 7, wherein, When the instruction is executed by the at least one processor, the electronic device: In the rich execution environment, the data for which the first noise value has been applied is calculated based on the weight value applied to the noise value.

9. The electronic device according to claim 5, wherein, When the instruction is executed by the at least one processor, the electronic device: In the trusted execution environment, a first value is generated based on the weight values, first noise values, and biases of at least a portion of the multiple layers of the artificial intelligence model; The generated first value is stored in a portion of the memory; as well as Output data is obtained based on calculations performed on a pre-stored first value and computational data obtained in the rich execution environment.

10. The electronic device according to claim 1, wherein, When the instruction is executed by the at least one processor, the electronic device: Identify the state of the electronic device, and The noise value is generated in the trusted execution environment when the state of the electronic device corresponds to the idle state.

11. The electronic device according to claim 10, wherein, When the instruction is executed by the at least one processor, the electronic device: Identify the number of layers in the plurality of layers of the artificial intelligence model that generate noise values; Based on the fact that the number of identified layers is less than a preset value, select a subset of layers from those that have never generated noise values; as well as Generate noise values ​​associated with the selected partial layers.

12. The electronic device according to claim 11, wherein, When the instruction is executed by the at least one processor, the electronic device: Suppress the generation of noise values ​​associated with the first layer of the plurality of layers of the artificial intelligence model.

13. The electronic device according to claim 1, wherein, When the instruction is executed by the at least one processor, the electronic device: When the event is identified, it is determined whether the noise value for at least a portion of the multiple layers of the artificial intelligence model is stored in the electronic device. When the electronic device does not store the noise value, in the trusted execution environment, calculations are performed on the data input to the at least portion of the plurality of layers based on the weight values ​​of the at least portion of the plurality of layers for which the noise value is not applied. When the noise value is stored in the electronic device, in the rich execution environment, the data input to the at least part of the multiple layers is calculated based on the weight value of the noise value applied to the at least part of the multiple layers.

14. A method of operating an electronic device, the electronic device comprising a memory and at least one processor, the at least one processor being configured to perform operations based on a plurality of execution environments, wherein, The plurality of execution environments includes rich execution environments and trusted execution environments, and the method includes: Noise values ​​are generated in the trusted execution environment; By multiplying the generated noise value with the weight values ​​of at least a portion of the multiple layers included in the artificial intelligence model stored in the electronic device in the trusted execution environment, the noise value is applied to the weight value to obtain a weight value with the noise value applied. When an event for executing the artificial intelligence model is identified, in the rich execution environment, computational data is obtained based on the computation of at least a portion of the data input to the plurality of layers using weights applied with the noise value; and In the trusted execution environment, output data is obtained based on the acquired computational data and the noise value applied to the weight values. Specifically, a first portion of the memory is allocated to the rich execution environment, and a second portion of the memory is allocated to the trusted execution environment. The artificial intelligence model is stored in a second portion of the memory, and this second portion is inaccessible to the at least one processor in the rich execution environment. The first portion of the memory is accessible to the at least one processor in both the rich execution environment and the trusted execution environment.

15. A non-transitory computer-readable storage medium for storing instructions, wherein, When executed by at least one processor of the electronic device, the instructions cause the electronic device to perform the method of claim 14.

Images