Electronic device, method and non-transitory storage medium for protecting image

Abstract

This document relates to an electronic device, a method, and a non-transient storage medium for image protection. The electronic device may include a processor comprising a processing circuit and a memory for storing instructions. When executed by the processor, the instructions may cause the electronic device to identify a first image among a plurality of images stored in the memory, identify a first region corresponding to a selected region of a protection target identified in the first image, distinguish between the first region and a second region remaining after excluding the selected region in the first image, set the first region as a designated noise feature vector, obtain a feature vector map in which the second region is set as an initial value, and obtain a second image to which the feature vector map is applied as a protection image for protecting the first image.

Description

Technology Field

[0001] This document relates to an electronic device, method, and non-transient storage medium for image protection. Background Technology

[0003] Various services and additional functions provided through electronic devices, such as portable electronic devices like smartphones, are gradually increasing. To enhance the utility value of these devices and satisfy the needs of diverse users, telecommunications service providers or electronic device manufacturers are competitively developing devices to offer various functions and differentiate themselves from competitors. Consequently, the various functions provided through electronic devices are also becoming increasingly sophisticated.

[0004] As technology advances, hostile attacks involving the unauthorized theft of personal images (e.g., photographs) are on the rise. As these attacks superimpose specific individuals' faces onto other images for unauthorized use, these faces are no longer protected. Hostile attacks pose a risk of causing misclassification or misrecognition during image classification, or damaging models used for image classification or generation (e.g., artificial intelligence models) by inserting malicious data. Recently, various technologies have been developed to prevent the unauthorized use of images resulting from hostile attacks.

[0005] The information described above may be provided as related art for the purpose of aiding understanding of this document. No claim or determination is made as to whether any of the foregoing may be applied as prior art related to this document. means of solving the problem

[0007] According to one embodiment, the electronic device may include a processor including a processing circuit and a memory for storing instructions.

[0008] According to one embodiment, the instructions may cause the electronic device to check a first image among a plurality of images stored in the memory when executed by the processor.

[0009] According to one embodiment, the instructions, when executed by the processor, may cause the electronic device to identify a first region corresponding to a selected region in the protected target identified in the first image.

[0010] According to one embodiment, the instructions, when executed by the processor, may cause the electronic device to distinguish the remaining second region excluding the selected region in the first region and the first image.

[0011] According to one embodiment, the instructions, when executed by the processor, may cause the electronic device to obtain a feature vector map in which the first region is set to a specified noise feature vector and the second region is set to an initial value.

[0012] According to one embodiment, the instructions, when executed by the processor, may cause the electronic device to acquire a second image to which the feature vector map has been applied as a protection image to protect the first image.

[0013] According to one embodiment, a method of operation in an electronic device may include an operation of checking a first image among a plurality of images stored in the memory of the electronic device.

[0014] According to one embodiment, the method may include an operation of identifying a first region corresponding to a selected region in a protected target identified in the first image.

[0015] According to one embodiment, the method may include an operation of distinguishing the remaining second region, excluding the selected region from the first region and the first image.

[0016] According to one embodiment, the method may include the operation of obtaining a feature vector map in which the first region is set to a specified noise feature vector and the second region is set to an initial value.

[0017] According to one embodiment, the method may include the operation of acquiring a second image to which the feature vector map is applied as a protection image for protecting the first image.

[0018] According to one embodiment, in a non-transient storage medium storing one or more programs, the program may include an instruction that causes the electronic device to perform an operation of identifying a first image among a plurality of images stored in the memory of the electronic device when executed by at least one processor of the electronic device.

[0019] According to one embodiment, the program may include an instruction that causes the electronic device to perform an operation of identifying a first region corresponding to a selected region in a protected target identified in the first image when executed by at least one processor of the electronic device.

[0020] According to one embodiment, the program may include instructions that, when executed by at least one processor of an electronic device, cause the electronic device to perform an operation of distinguishing the remaining second region excluding the selected region from the first region and the first image.

[0021] According to one embodiment, the program may include an instruction that, when executed by at least one processor of an electronic device, causes the electronic device to execute an operation of acquiring a feature vector map in which the first region is set to a specified noise feature vector and the second region is set to an initial value.

[0022] According to one embodiment, the program may include instructions that, when executed by at least one processor of an electronic device, cause the electronic device to perform an operation of acquiring a second image to which the feature vector map is applied as a protection image for protecting the first image. Brief explanation of the drawing

[0024] FIG. 1 is a schematic diagram of an electronic system according to one embodiment. FIG. 2 is a diagram showing the configuration of an electronic device according to one embodiment. FIG. 3 is a block diagram illustrating an example of generating a protective image in an electronic device according to one embodiment. FIG. 4a is a drawing illustrating an example of generating a protective image in an electronic device according to one embodiment. FIG. 4b is a drawing showing an example of a protective image in an electronic device according to one embodiment. FIG. 5 is a diagram illustrating a method of operation in an electronic device according to one embodiment. FIG. 6 is a diagram illustrating a method of operation in an electronic device according to one embodiment. FIG. 7 is a diagram illustrating an example of a screen for setting the protection strength of a protection image in an electronic device according to one embodiment. FIG. 8 is a block diagram of a generative artificial intelligence system according to one embodiment. FIG. 9 is a block diagram of an AI framework according to one embodiment. In relation to the description of the drawings, the same or similar reference numerals may be used for identical or similar components. Specific details for implementing the invention

[0025] Hereinafter, embodiments of this document are described in detail with reference to the drawings so that those skilled in the art can easily implement them. However, this document may be implemented in various different forms and is not limited to the embodiments described herein. In relation to the description of the drawings, identical or similar reference numerals may be used for identical or similar components. Additionally, in the drawings and related descriptions, descriptions of well-known functions and configurations may be omitted for clarity and brevity. As used in the embodiments of this document, the term "user" may refer to a person using an electronic device or a device using an electronic device (e.g., an artificial intelligence electronic device).

[0026] Furthermore, the term "user query" as used in this document may be used to mean a user making a request to an electronic device by voice, but it may also be understood as or substituted with the meanings of "user request" or "user command." In other words, it is obvious to those skilled in the art that, in order for a user to obtain a response or guidance, etc. from an electronic device, it is possible to use sentences in the form of commands or requests, not just sentences in the form of queries or inquiries.

[0027] Furthermore, before the user first provides a query or command, the electronic device may determine whether a specific task is in progress or scheduled to be performed based on an image captured through a video input device (e.g., a camera), and based on the determination result, the electronic device may first query the user and receive the user's query, request, or command in the form of an answer to provide guidance for performing the task.

[0028] FIG. 1 is a block diagram of an electronic device (101) in a network environment (100) according to various embodiments. Referring to FIG. 1, in the network environment (100), the electronic device (101) may communicate with an electronic device (102) through a first network (198) (e.g., a short-range wireless communication network) or may communicate with at least one of an electronic device (104) or a server (108) through a second network (199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (101) may communicate with the electronic device (104) through a server (108). According to one embodiment, the electronic device (101) may include a processor (120), memory (130), input module (150), sound output module (155), display module (160), audio module (170), sensor module (176), interface (177), connection terminal (178), haptic module (179), camera module (180), power management module (188), battery (189), communication module (190), subscriber identification module (196), or antenna module (197). In some embodiments, at least one of these components (e.g., connection terminal (178)) may be omitted from the electronic device (101), or one or more other components may be added. In some embodiments, some of these components (e.g., sensor module (176), camera module (180), or antenna module (197)) may be integrated into a single component (e.g., display module (160)).

[0029] The processor (120) can control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) by executing software (e.g., a program (140)), and can perform various data processing or operations. According to one embodiment, as at least part of the data processing or operations, the processor (120) can store commands or data received from other components (e.g., a sensor module (176) or a communication module (190)) in volatile memory (132), process the commands or data stored in volatile memory (132), and store the resulting data in non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or an auxiliary processor (123) that can operate independently or together with it (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor). For example, if the electronic device (101) includes a main processor (121) and an auxiliary processor (123), the auxiliary processor (123) may be configured to use lower power than the main processor (121) or to be specialized for a designated function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as part thereof.

[0030] The auxiliary processor (123) may control at least some of the functions or states associated with at least one component of the electronic device (101) (e.g., display module (160), sensor module (176), or communication module (190)) on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. According to one embodiment, the auxiliary processor (123) (e.g., image signal processor or communication processor) may be implemented as part of another functionally related component (e.g., camera module (180) or communication module (190)). According to one embodiment, the auxiliary processor (123) (e.g., neural network processing unit) may include a hardware structure specialized for processing an artificial intelligence model. The artificial intelligence model may be generated through machine learning. Such learning may be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or through a separate server (e.g., server (108)). The learning algorithm may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model may include a plurality of artificial neural network layers.An artificial neural network may be a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to the hardware structure, the artificial intelligence model may include a software structure, either additionally or substantially.

[0031] The memory (130) can store various data used by at least one component of the electronic device (101) (e.g., processor (120) or sensor module (176)). The data may include, for example, input data or output data for software (e.g., program (140)) and related commands. The memory (130) may include volatile memory (132) or non-volatile memory (134).

[0032] The program (140) may be stored as software in memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0033] The input module (150) can receive commands or data to be used for a component of the electronic device (101) (e.g., processor (120)) from outside the electronic device (101) (e.g., user). The input module (150) may include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0034] The sound output module (155) can output a sound signal to the outside of the electronic device (101). The sound output module (155) may include, for example, a speaker or a receiver. The speaker may be used for general purposes, such as multimedia playback or recording playback. The receiver may be used to receive incoming calls. According to one embodiment, the receiver may be implemented separately from the speaker or as part thereof.

[0035] The display module (160) can visually provide information to an external (e.g., user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling said device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of the force generated by said touch.

[0036] The audio module (170) can convert sound into an electrical signal or, conversely, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150) or output sound through the sound output module (155) or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphones) connected directly or wirelessly to the electronic device (101).

[0037] The sensor module (176) can detect the operating state of the electronic device (101) (e.g., power or temperature) or the external environmental state (e.g., user state) and generate an electrical signal or data value corresponding to the detected state. According to one embodiment, the sensor module (176) may include, for example, a gesture sensor, a gyroscope sensor, a barometric pressure sensor, a magnetic sensor, an accelerometer sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biosensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0038] The interface (177) may support one or more specified protocols that can be used for the electronic device (101) to be connected directly or wirelessly to an external electronic device (e.g., electronic device (102)). According to one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0039] The connection terminal (178) may include a connector through which the electronic device (101) can be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0040] The haptic module (179) can convert an electrical signal into a mechanical stimulus (e.g., vibration or movement) or an electrical stimulus that can be perceived by the user through tactile or kinesthetic senses. According to one embodiment, the haptic module (179) may include, for example, a motor, a piezoelectric element, or an electric stimulation device.

[0041] The camera module (180) can capture still images and video. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0042] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented, for example, as at least part of a power management integrated circuit (PMIC).

[0043] The battery (189) can supply power to at least one component of the electronic device (101). According to one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0044] The communication module (190) can support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between an electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may include one or more communication processors that operate independently of the processor (120) (e.g., application processor) and support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., cellular communication module, short-range wireless communication module, or GNSS (global navigation satellite system) communication module) or a wired communication module (194) (e.g., LAN (local area network) communication module, or power line communication module). The corresponding communication module among these communication modules can communicate with an external electronic device (104) through a first network (198) (e.g., a short-range communication network such as Bluetooth, WiFi (wireless fidelity) direct, or IrDA (infrared data association)) or a second network (199) (e.g., a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules may be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can identify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) using subscriber information (e.g., International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module (196).

[0045] The wireless communication module (192) can support 5G networks and next-generation communication technologies following 4G networks, for example, new radio access technology. NR access technology can support high-speed transmission of high-capacity data (enhanced mobile broadband (eMBB)), minimization of terminal power and connection of multiple terminals (massive machine type communications (mMTC)), or high reliability and low latency (ultra-reliable and low-latency communications (URLLC)). The wireless communication module (192) can support a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate, for example. The wireless communication module (192) can support various technologies for securing performance in the high-frequency band, such as beamforming, massive MIMO (multiple-input and multiple-output), full-dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large-scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), external electronic device (e.g., electronic device (104)), or network system (e.g., second network (199)). According to one embodiment, the wireless communication module (192) can support a Peak data rate (e.g., 20 Gbps or more) for realizing eMBB, loss coverage (e.g., 164 dB or less) for realizing mMTC, or U-plane latency (e.g., downlink (DL) and uplink (UL) each 0.5 ms or less, or round trip 1 ms or less) for realizing URLLC.

[0046] An antenna module (197) can transmit a signal or power to or from an external source (e.g., an external electronic device). According to one embodiment, the antenna module (197) may include an antenna comprising a radiator made of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). According to one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as a first network (198) or a second network (199), may be selected from the plurality of antennas, for example, by a communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device through the selected at least one antenna. According to some embodiments, in addition to the radiator, other components (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as part of the antenna module (197).

[0047] According to various embodiments, the antenna module (197) may form a mmWave antenna module. According to one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent to a first surface (e.g., bottom surface) of the printed circuit board and capable of supporting a specified high frequency band (e.g., mmWave band), and a plurality of antennas (e.g., array antennas) disposed on or adjacent to a second surface (e.g., top surface or side surface) of the printed circuit board and capable of transmitting or receiving a signal of the specified high frequency band.

[0048] At least some of the above components can be connected to each other via a communication method between peripheral devices (e.g., bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)) and exchange signals (e.g., commands or data) with each other.

[0049] According to one embodiment, commands or data may be transmitted or received between an electronic device (101) and an external electronic device (104) through a server (108) connected to a second network (199). Each of the external electronic devices (102, or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations performed on the electronic device (101) may be performed on one or more of the external electronic devices (102, 104, or 108). For example, if the electronic device (101) needs to perform a function or service automatically or in response to a request from a user or another device, the electronic device (101) may request one or more external electronic devices to perform at least part of the function or service instead of performing the function or service itself or additionally. One or more external electronic devices that receive the above request may execute at least part of the requested function or service, or additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may provide the result as is or additionally processed as at least part of the response to the request. For this purpose, for example, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used. The electronic device (101) may provide ultra-low latency services using, for example, distributed computing or mobile edge computing. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server using machine learning and / or neural networks. According to one embodiment, the external electronic device (104) or the server (108) may be included within a second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0050] The number of processors (120) may be one or more. For example, the processor (120) may have the structure of a multi-core processor such as a dual core, a quad core, or a hexa core.

[0051] The processor (120) can control the operations of the electronic device (101) by executing instructions stored in memory (130). For example, the processor (120) may correspond to a plurality of processors that divide and collectively perform a plurality of operations among the processors.

[0053] FIG. 2 is a diagram showing the configuration of an electronic device according to one embodiment.

[0054] Referring to FIG. 2, an electronic device (101) according to one embodiment (e.g., the electronic device (101) of FIG. 1) may include a processor (210), a memory (220), and a display (230). The electronic device (201) is not limited thereto and may be configured to include various other components or to exclude some of the components.

[0055] According to one embodiment, a processor (210) (e.g., processor (120) of FIG. 1) can identify (e.g., acquire) a first image (e.g., original image) among a plurality of images stored in memory (220). The processor (201) can execute a function (e.g., application) for creating a protection image to protect a protection target identified in the first image.

[0056] According to one embodiment, the processor (210) classifies objects included in a first image, identifies an object corresponding to a protected target (e.g., person, animal, or object) among the classified objects, and selects (e.g., classifies) the entire area or a part of the identified object as an area to be protected. The processor (210) can identify a first area (e.g., an area corresponding to a face) corresponding to an area selected from the protected target (e.g., a specific person) (e.g., face segmentation) to protect the artificial intelligence model performing the manipulation from recognizing and manipulating a specific person when the protected target is manipulated by an adversarial attack. The processor (210) can classify (e.g., classifies) the remaining area excluding the first area and the area selected from the first image as a second area.

[0057] According to one embodiment, the processor (210) may acquire (e.g., generate) a feature vector map in which a first region is set to a specified noise feature vector and a second region is set to an initial value. According to one embodiment, the processor (210) may acquire (e.g., use) a result image (e.g., a second image) to which the feature vector map has been applied as a protection image to protect the first image. The processor (210) may acquire the second image as a protection image based on identifying that the specified noise feature vector set in the first region matches the target value of the first region. Based on identifying that the specified noise feature vector set in the first region does not match the target value of the first region, the processor (210) may update the specified feature vector until it matches the target value, update the feature vector map by setting the first region to the updated noise feature vector and the second region to an initial value, and update the second image by reflecting the updated feature vector map. The processor (210) may acquire the updated second image as a protection image. Here, the specified noise feature vector may be noise added to the pixels of a first region (e.g., the face of a specific person) of a first image to disrupt an artificial intelligence model that manipulates a protected target with an adversarial attack. The specified noise feature vector may be designated as noise values ​​generated based on a first loss value for verifying similarity between the first image and the second image with added noise, and a second loss value for verifying the difference in image quality between the first image and the second image, and may be updated until it matches a target value. The target value of the first region may refer to a target noise feature vector (e.g., a noise delta value) that reflects the noise value for updating the specified noise feature vector based on a minimum value at which similarity between the first image and the second image decreases and a minimum value at which image quality loss decreases.

[0059] FIG. 3 is a block diagram illustrating an example of generating a protection image in an electronic device according to one embodiment, FIG. 4a is a drawing illustrating an example of generating a protection image in an electronic device according to one embodiment, and FIG. 4b is a drawing illustrating an example of a protection image in an electronic device according to one embodiment. In FIG. 3, FIG. 4a, and FIG. 4b, the protected object is illustrated as a specific person as an example, but is not limited thereto, and the protected object may be a designated animal or a designated object.

[0060] Referring to FIGS. 2, FIGS. 3, FIGS. 4a and FIGS. 4b, a processor (210) (e.g., processor (120) of FIG. 2) of an electronic device (101) (e.g., electronic device (101) of FIG. 1) according to one embodiment acquires a first image (301) using a classification module (310), identifies a first region (e.g., face region of a specific person) corresponding to a selected region from a protected target identified in the first image (301) in an initial operation, and can distinguish (e.g., classify) a second region which is the remaining region excluding the first region and the selected region in the first image. The processor (210) performs an operation (e.g., face segmentation) of classifying the first image (301) into a first region (e.g., face region) and a second region (e.g., background region) in an initial operation (e.g., before acquiring a feature vector map), and the image (G(x) acquired through the classification operation u A selected region image (SR(x) obtained by binarizing )) and adding noise to the first region u ))(303)(e.g., mask image) can be acquired. The processor (210) can acquire a selected region image (SR(x u In ))(303), a selection area map (410) can be obtained by setting a maximum noise value (e.g., 1) in the first area (311) and an initial value (e.g., 0) in the remaining area, the second area (312).

[0061] According to one embodiment, the processor (210) may transmit a first image (301) as an image to be input to the feature vector module (320) in an initial operation. The processor (210) extracts feature vectors (e.g., face recognition result vector values) from the input image using the feature vector module (320), and a first loss value (e.g., perception loss, Loss) by similarity verification based on the average value generated from the extracted feature vectors through the similarity verification module (330). p ) can be checked. The processor (210) uses the image quality verification module (340) to check the first image ( )(301) and the second image( A second loss value (Loss) indicating the image quality loss value by checking the difference in image quality between )(303) q ) can be verified. According to one embodiment, the processor (210) assigns weights to the first loss value and the second loss value, respectively, to obtain a total loss value (Loss) representing the multiplied sum between the first loss value and the second loss value. total You can check ).

[0062] According to one embodiment, the processor (210) can use a protection image generation module (350) to identify (e.g., generate or calculate) new noise based on a first loss value and a second loss value or based on a total loss value, and can generate a second map (e.g., a noise feature vector map in which new noise values ​​are set for each pixel of the entire area) (420) in which the identified new noise is set for a first area and a second area. According to one embodiment, the processor (210) can use the protection image generation module (350) to combine a first map (e.g., a selection area map) (410) and a second map (420) to generate a third map (e.g., a feature vector map) (430) in which a noise feature vector designated for the first area is set and an initial value is set for the second area. A designated noise feature vector (n adv ) is the target value (e.g., minimum value( ) Adjust or update in a direction that reduces similarity and minimizes image quality loss (e.g., )can be.

[0063] According to one embodiment, the processor (210) uses a protection image generation module (350) to create a third map ( )(430)(e.g., image (401) corresponding to the third map) as the first image ( Create a second image (302) by reflecting it in )(301) ( ) can.

[0064] According to one embodiment, if the processor (210) identifies that a noise feature vector specified in a first region reflected in the second image (302) matches a target value using a protection image generation module (350), the second image (302) can be acquired as a protection image (403) for protecting the first image (301).

[0065] According to one embodiment, if the processor (210) identifies that the noise feature vector specified in the first region reflected in the updated second image (302) using the protection image generation module (350) does not match the target value, the second image (302) can be transmitted to the feature vector module (320) as an input image to generate a protection image.

[0066] According to one embodiment, the processor (210) can extract feature vectors from the second image (302) using a feature vector module (320). According to one embodiment, the processor (210) can check similarity based on the extracted feature vectors using a similarity check module (330) and check a first loss value (perception loss) corresponding to the checked similarity. According to one embodiment, the processor (210) can check the difference in quality between the first image (301) and the second image (302) using a quality check module (340) and check a second loss value (quality loss) representing the quality loss according to the checked difference in quality.

[0067] According to one embodiment, the processor (210) can generate new noise based on a first loss value and a second loss value or a total loss value using a protection image generation module (350) as described in the initial operation, generate a second map (420) with the new noise added, set an updated noise feature vector that indicates the new noise value only in the first region by reflecting the first map (410) in the second map (420), and update a third map (430) by setting an initial value (e.g., 0) in the second region. The processor (201) can update the second image (302) by reflecting the updated third map (430) using the protection image generation module (350). According to one embodiment, if the processor (210) identifies that a noise feature vector set in a first region reflected in the second image (302) matches a target value using a protection image generation module (350), the second image (302) can be acquired as a protection image (403) for protecting the first image (301).

[0068] According to one embodiment, if the processor (210) identifies that the noise feature vector set in the first area of ​​the updated second image does not match the target value, it may repeatedly perform the operation of updating the noise feature vector until the noise feature vector set in the first area matches the target value of the first area, and then setting the updated noise feature vector in the first area to update the second image. According to one embodiment, the processor (210) may update the noise feature vector to be set in the first area (e.g., noise values ​​to be added to the pixels of the first area) by adjusting (e.g., updating) it in a direction in which similarity decreases based on the first loss value, and by adjusting (e.g., updating) it in a direction in which image quality loss decreases based on the second loss value, and, as shown in FIG. 4, update (e.g., generate) the third map (430) in which the updated noise feature vector is set in the first area. The processor (210) may repeatedly perform the operation of updating the second image to obtain (e.g., use) the second image (203) reflecting the updated third map (430) as a protection image. Here, the protection image (403) may have noise added to it, representing an updated noise feature vector in an area corresponding to the selection area of ​​the first image (e.g., eye area (403-1)), as shown in FIG. 4b.

[0069] According to one embodiment, the processor (210) may display a screen for a function (e.g., an application) for setting a protection image in an electronic device (201) on a display (230) (e.g., the display (160) of FIG. 1). According to one embodiment, the processor (210) may display a first screen on the display (230) for setting a protection strength to generate a protection image. For example, objects for setting the protection strength may be displayed as objects representing high, medium, and low. Not limited thereto, objects for setting the protection strength may be displayed as objects representing a numerical value or level designated as the protection strength. The designated protection strength may be designated differently based on electronic device product characteristics, display characteristics, or characteristics of a module (e.g., an AI model) for generating a protection image.

[0070] According to one embodiment, the processor (210) can acquire a protection image to protect against hostile attacks when transmitting the first image (301) to an external electronic device or using it for content.

[0071] According to one embodiment, the processor (210) may be a hardware module or a software module (e.g., an application program) and may be a hardware component (function) or a software element (program) comprising at least one of various sensors provided in the electronic device (201), an input / output interface, a module for managing the state or environment of the electronic device (201), or a communication module.

[0072] According to one embodiment, the processor (210) may include, for example, one or more combinations of hardware, software, or firmware. The processor (210) may be configured to omit at least some of the components or to include additional components for performing image processing operations in addition to the components.

[0073] According to one embodiment, the memory (220) (e.g., the memory (130) of FIG. 1) may store information and / or data associated with the operation of the electronic device (201). According to one embodiment, the memory (220) may store at least one instruction (or command) that causes at least one operation of the electronic device (201). When executed by the processor (210), the at least one instruction may cause the electronic device (201) to perform the corresponding operation.

[0074] According to one embodiment, the memory (220) can store various data generated during program execution, including a program used for functional operation (e.g., the program (140) of FIG. 1). The memory (240) may largely include a program area (e.g., the program (140) of FIG. 1) and a data area (not shown). The program area (140) may store related program information for operating the electronic device (201), such as an operating system (OS) (e.g., the operating system (142) of FIG. 1) that boots the electronic device (201). The data area (not shown) may store transmitted and / or received data and generated data according to one embodiment. Additionally, the memory (220) may be configured to include at least one storage medium among flash memory, hard disk, multimedia card micro type memory (e.g., secure digital (SD) or extreme digital (XD) memory), RAM, and ROM. According to one embodiment, the memory (220) may store information related to a screen displayed on a display and information related to pixels used to modify the screen (e.g., modulation parameters). In addition, the memory (220) may store information necessary to modify the screen.

[0075] According to one embodiment, the display (230) may display a screen related to a function executed in the electronic device (201). According to one embodiment, the display (230) may display a setting screen (e.g., an editing screen) for screen modification. According to one embodiment, the display (240) may be implemented in the form of a touch screen. When the display (240) is implemented in the form of a touch screen together with an input module, it may display various information generated according to the user's touch actions. According to one embodiment, the display (240) may be composed of at least one of an LCD (liquid crystal display), TFT-LCD (thin film transistor LCD), OLED (organic light emitting diodes), LED, AMOLED (active matrix organic LED), micro LED, mini LED, flexible display, and 3-dimensional display. Additionally, some of these displays may be configured to be transparent or light-transmitting so that the outside can be seen through them. This can be configured in the form of a transparent display including a TOLED (transparent OLED). According to one embodiment, in addition to the display (230), it may further include other display modules mounted thereon (e.g., an extended display or a flexible display).

[0076] An electronic device according to one embodiment (e.g., the electronic device (101) of FIG. 1 or the electronic device (201) of FIG. 2) may implement a related software module (e.g., the program (140) of FIG. 1) for performing a function for image protection. The memory of the electronic device (e.g., the memory (130) of FIG. 1 and / or the memory (220) of FIG. 2) may store instructions (e.g., instructions) to implement the software module. At least one processor (e.g., the processor (120) of FIG. 1 and / or the processor (210) of FIG. 2) may execute the instructions stored in memory to implement the software module and may control hardware associated with the function of the software module (e.g., the sensor module (176) of FIG. 1, the camera module (180), the communication module (190) of FIG. 1, the display module (160) of FIG. 1 or the display (230) of FIG. 2).

[0077] A software module of an electronic device according to one embodiment may be configured to include a kernel (or HAL), a framework (e.g., middleware (144) of FIG. 1), and an application (e.g., application (146) of FIG. 1). At least some of the software modules may be preloaded onto the electronic device or downloadable from a server (e.g., server (108)).

[0078] According to one embodiment, the kernel may include, for example, a system resource manager or a device driver, but may be configured to include other modules, not limited thereto. The system resource manager may perform control, allocation, or reclamation of system resources. The device driver may include, for example, a display driver, a camera driver, a Bluetooth driver, a shared memory driver, a USB driver, a keypad driver, a WIFI driver, an audio driver, or an IPC (inter-process communication) driver.

[0079] According to one embodiment, the framework may provide various functions to an application through an application programming interface (API) (not shown) to provide functions commonly required by the application or to enable the application to efficiently use limited system resources within the electronic device. The framework may include modules that form combinations of various functions of the components. The framework may provide modules specialized for each type of operating system to provide differentiated functions. The framework may dynamically delete some existing components or add new components.

[0080] According to one embodiment, the application may be configured to include an application (e.g., a module, a manager, or a program) for displaying an image of the external environment in real space. The application may include an application received from an external electronic device (e.g., a server (108) or an electronic device (102, 104)). According to one embodiment, the application may include a preloaded application or a third-party application downloadable from a server. The components of the software module and the names of the components according to the illustrated embodiments may vary depending on the type of operating system. According to one embodiment, at least a portion of the software module may be implemented as software, firmware, hardware, or a combination of at least two of these. At least a portion of the software module may be implemented (e.g., executed) by a processor (e.g., AP). At least a portion of the software module may include, for example, a module, a program, a routine, a set of instructions, or a process for performing at least one function.

[0081] As such, in one embodiment, the main components of an electronic device have been described through the electronic device (101, 201) of FIGS. 1 and 2. However, in various embodiments, the components illustrated in FIGS. 1 and 2 are not all essential components, and the electronic device (101, 201) may be implemented with more components than those illustrated, or with fewer components. Additionally, the positions of the main components of the electronic device (101, 201) described above through FIGS. 1 and 2 may be changed according to various embodiments.

[0083] According to one embodiment, an electronic device (e.g., the electronic device (101) of FIG. 1 or the electronic device (201) of FIG. 2) may include a processor including a processing circuit (e.g., the processor (120) of FIG. 1 or the processor (210) of FIG. 2)) and a memory for storing instructions (e.g., the memory (130) of FIG. 1 or the memory (220) of FIG. 2).

[0084] According to one embodiment, the instructions may cause the electronic device to check a first image among a plurality of images stored in the memory when executed by the processor.

[0085] According to one embodiment, the instructions, when executed by the processor, may cause the electronic device to identify a first region corresponding to a selected region in the protected target identified in the first image.

[0086] According to one embodiment, the instructions, when executed by the processor, may cause the electronic device to distinguish the remaining second region excluding the selected region in the first region and the first image.

[0087] According to one embodiment, the instructions, when executed by the processor, may cause the electronic device to obtain a feature vector map in which the first region is set to a specified noise feature vector and the second region is set to an initial value.

[0088] According to one embodiment, the instructions, when executed by the processor, may cause the electronic device to acquire a second image to which the feature vector map has been applied as a protection image to protect the first image.

[0089] According to one embodiment, the instructions may cause the electronic device, when executed by the processor, to acquire the second image as the protection image based on identifying that the designated noise feature vector set in the first region matches the target value of the first region.

[0090] According to one embodiment, the instructions may cause the electronic device, when executed by the processor, to update the specified noise feature vector until it matches the target value based on identifying that the specified noise feature vector set in the first region does not match the target value of the first region, set the first region to the updated noise feature vector and set the second region to an initial value to update the feature vector map, and update the second image by reflecting the updated feature vector map.

[0091] According to one embodiment, the instructions, when executed by the processor, may cause the electronic device to check a first loss value for checking the similarity between a noise feature vector extracted from the second image and a feature vector of the first image, check a second loss value for checking the quality loss between the first image and the second image, and update the designated noise feature vector set in the first region with a noise specific vector generated based on the first loss value and the second loss value.

[0092] According to one embodiment, the target value of the first region may be a target noise feature vector that reflects a preset noise value for updating the specified noise feature vector based on a minimum value at which the similarity between the first image and the second image is reduced and a minimum value at which the loss of image quality is reduced.

[0093] According to one embodiment, the instructions, when executed by the processor, may cause the electronic device to obtain a total loss value as a multiplied sum between the first loss value and the second loss value, and to generate the noise value for specifying the specified noise feature vector based on the total loss value.

[0094] According to one embodiment, the instructions, when executed by the processor, may cause the electronic device to distinguish the first region and the second region in the first image and obtain a selection region map before obtaining the feature vector map.

[0095] According to one embodiment, the selection area map may have a maximum value of 1 set in the first area and an initial value of 0 set in the second area.

[0096] According to one embodiment, the instructions, when executed by the processor, may cause the electronic device to display a first screen on the display of the electronic device, the first screen including objects for setting a protection strength for protecting the first image.

[0097] According to one embodiment, the instructions, when executed by the processor, may cause the electronic device to set the target value based on the protection strength of the selected object, based on the selection of one of the objects for setting the protection strength through the first screen.

[0099] FIG. 5 is a diagram illustrating an example of an operation method in an electronic device according to one embodiment. In the following embodiments, each operation may be performed sequentially, but is not necessarily performed sequentially. For example, the order of each operation may be changed, and at least two operations may be performed in parallel.

[0100] Referring to FIG. 5, in operation 501, an electronic device according to one embodiment (e.g., the electronic device (101) of FIG. 1 and the electronic device (201) of FIG. 2) can identify (e.g., acquire) a first image that is to be protected (e.g., the first image (301) of FIG. 3 or the original image) among a plurality of images stored in a memory (e.g., the memory (130) of FIG. 1 or the memory (220) of FIG. 2).

[0101] In operation 503, according to one embodiment, the electronic device may classify objects included in a first image, identify an object corresponding to a protected target (e.g., person, animal, or object) among the classified objects, identify a selected area (hereinafter referred to as the first area) for protecting the entire area or a part area of ​​the identified object, and classify the remaining area (hereinafter referred to as the second area) excluding the first area and the selected area from the first image. According to one embodiment, the electronic device may identify a specific person as a protected target and classify an area corresponding to the face of the specific person into a selected first area for adding noise (e.g., face segmentation). The electronic device according to one embodiment may generate a selected area map (hereinafter referred to as the first map) in which a maximum value (e.g., 1) is set in the classified first area and an initial value (e.g., 0) is set in the classified second area, and obtain a mask image reflecting the first map. The operation of generating such a first map may be performed before generating the protected image in the initial operation.

[0102] In operation 505, an electronic device according to one embodiment may obtain a feature vector map (hereinafter referred to as a third map) in which a first region is set to a specified noise feature vector (e.g., a noise delta value) and a second region is set to an initial value (e.g., 0). The electronic device may obtain a first loss value and a second loss value or a total loss value to add noise to the second image in a direction in which the similarity to the feature vectors extracted from the second image decreases and the loss of image quality due to the difference in image quality decreases, and may identify (e.g., calculate) the noise to be added to the feature vectors based on the first loss value and the second loss value or the total loss value, and obtain a second map (e.g., a noise feature vector map) that is adjusted (e.g., updated) to feature vectors with added noise (e.g., noise feature vectors). The electronic device may generate a third map in which a noise feature vector specified in the first region is set and an initial value is set in the second region by reflecting the second map in the first map. The electronic device can obtain a second image to which a third map has been applied (e.g., a second image with added noise). Here, the second image may be the same as the first image without added noise in the initial operation, and in the next operation, it may be an image with added noise to the specified noise feature vectors set in the first region (e.g., an updated image). The electronic device may repeat operation 505 to update the second image by adding noise in a direction that reduces similarity and reduces image quality loss until the specified noise feature vectors set in the first region reach a target value (e.g., match).

[0103] In operation 507, an electronic device according to one embodiment may acquire (e.g., use) a second image to which a third map has been applied as a protective image to protect the first image. According to one embodiment, if the designated noise feature vectors set in the first area of ​​the third map match the target value, the electronic device may acquire the second image to which the third map has been applied as a protective image to protect the first image. If the designated noise feature vectors do not match the target value, the electronic device according to one embodiment may repeat operations 505 and 507 until the designated noise feature vectors match the target value.

[0105] FIG. 6 is a diagram illustrating an example of an operation method in an electronic device according to one embodiment. In the following embodiments, each operation may be performed sequentially, but is not necessarily performed sequentially. For example, the order of each operation may be changed, and at least two operations may be performed in parallel.

[0106] Referring to FIG. 6, in operation 601, an electronic device according to one embodiment (e.g., the electronic device (101) of FIG. 1 and the electronic device (201) of FIG. 2) can identify (e.g., acquire) a first image that is to be protected (e.g., the first image (301) of FIG. 3 or the original image) among a plurality of images stored in a memory (e.g., the memory (130) of FIG. 1 or the memory (220) of FIG. 2).

[0107] In operation 603, according to one embodiment, the electronic device may classify objects included in a first image, identify an object corresponding to a protected target (e.g., person, animal, or object) among the classified objects, identify a selected area (hereinafter referred to as the first area) for protecting the entire area or a part area of ​​the identified object, and classify the remaining area (hereinafter referred to as the second area) excluding the first area and the selected area from the first image. The first area may be classified as a selected area for adding noise (e.g., face segmentation). The electronic device according to one embodiment may generate a selected area map (hereinafter referred to as the first map) by setting a maximum value (e.g., 1) in the classified first area and setting an initial value (e.g., 0) in the classified second area, and acquire a mask image reflecting the first map. The operation of generating such a first map may be performed before generating the protected image in the initial operation.

[0108] In operation 605, an electronic device according to one embodiment may extract a feature vector (or a plurality of feature vectors) from a second image. Since operation 605 is an initial operation before performing an operation to add noise, the second image may be an image identical to the first image (e.g., original image) and may be an image without added noise.

[0109] In operation 607, an electronic device according to one embodiment can identify a first loss (perception loss) (e.g., a first loss value) corresponding to the similarity between a first image and a second image based on feature vectors extracted from a second image. An electronic device according to one embodiment can identify a second loss (quality loss) (e.g., a second loss value) representing the quality loss of the second image relative to the first image based on a comparison of the first image and the second image. An electronic device according to one embodiment can obtain a total loss (e.g., a total loss value) based on the first loss (e.g., a first loss value) and the second loss (e.g., a second loss value). For example, the total loss value may be the sum of the multipliers of the first loss value and the second loss value.

[0110] 609 In operation, an electronic device according to one embodiment acquires (generates) noise to be added to a second image based on a first loss value and a second loss value or a total loss value, and acquires (e.g., generates) a second map (e.g., a noise feature vector map) in which the acquired noise is reflected in the feature vectors of the second image.

[0111] In operation 611, an electronic device according to one embodiment may obtain (e.g., generate) a feature vector map (hereinafter referred to as a third map) in which a first region is set to a specified noise feature vector (e.g., noise delta value) by reflecting a second map on a first map, and a second region is set to an initial value (e.g., 0). An electronic device according to one embodiment may obtain an updated second image (e.g., a second image with added noise) reflecting the third map.

[0112] In operation 613, an electronic device according to one embodiment can check whether specified noise feature vectors set in a first region match a target value. If the result of the check is that they do not match, operation 615 is performed, and if they match, operation 617 is performed.

[0113] In operation 615, an electronic device according to one embodiment may extract feature vectors from a second image (e.g., a second image with added noise) that reflects a third map. Subsequently, the electronic device may perform operation 607. The electronic device may repeat operations 607 to 613 to update the second image by adding noise in a direction that reduces similarity and reduces image quality loss until the specified noise feature vectors set in the first region match the target value.

[0114] 617 In operation, an electronic device according to one embodiment may acquire a second image reflecting a third map as a protection image. Afterward, the electronic device may terminate the operation.

[0116] FIG. 7 is a diagram illustrating an example of a screen for setting the protection strength of a protection image in an electronic device according to one embodiment.

[0117] Referring to FIG. 7, according to one embodiment, an electronic device (201) (e.g., the electronic device (101) of FIG. 1 or the electronic device (201) of FIG. 2) may display a first screen (701) containing an object (710) for setting protection strength through a display (230) (the display module (160) of FIG. 1 or the display (230) of FIG. 2). According to one embodiment, the electronic device (201) may display a first image (e.g., an original image) (703) on the first screen (701). For example, the object (710) for setting protection strength may include objects representing high, medium, and low. Not limited thereto, the objects for setting protection strength may be displayed as objects representing a numerical value or level designated as protection strength. Here, the protection strength may be designated differently based on the characteristics of the electronic device product, the display characteristics, or the characteristics of the model (e.g., an AI model).

[0118] According to one embodiment, the electronic device (201) may specify a target value (e.g., a first target value and a second target value, or a total target value representing the sum of the scales between the first target value and the second target value) for generating new noise in a direction in which similarity is reduced and image quality loss is reduced based on the selected protection strength when the user selects one of the protection strengths of upper, middle, and lower included in an object (710) for setting the protection strength displayed on the first screen (701). Here, the first screen (701) may include an object (710) for setting the protection strength and an object (not shown) for specifying a selection area (e.g., a first area) in a first image (e.g., an original image). For example, the electronic device may display a first image on the first screen (701) and select an area specified by the user in the displayed first image as the first area. For example, the electronic device may display information about objects classified in the first image (e.g., a specific person's face, an animal's face, or an object) on the first screen (701), and select an area corresponding to the selected object as the first area through the information about the objects. For example, the electronic device may automatically classify the first image through a classification module (e.g., the classification module (310) of FIG. 3) to select a first area to add noise.

[0119] According to one embodiment, if the electronic device (201) identifies that a selection area (e.g., face area) and a protection strength (e.g., strong) are set through the first screen (601) before performing the operation to generate a protection image, when performing the operation to generate a protection image, an image (e.g., an image obtained through a GAN(G) model) for classifying (face segmentation) the set selection area (SR) (e.g., face area) )(e.g., mask image (331) of FIG. 3)) can be obtained, and a target value for generating new noise (e.g., a first target value and a second target value or a total target value representing the sum of the scales between the first target value and the second target value) can be determined.

[0120] According to one embodiment, the electronic device (201) may further include objects capable of selecting face area regions (e.g., eye, nose, mouth, and ear regions) on an object for selecting a first region on a first screen (601). According to one embodiment, the electronic device (201) may designate a selection region (SR) (e.g., first region) to add noise by selecting at least one face area region among the face area regions (e.g., eye, nose, mouth, and ear regions) and setting the intensity of the face area desired by the user differently.

[0121] According to one embodiment, when the electronic device (201) identifies that a plurality of face region areas (e.g., eyes and nose) among face region areas (e.g., eyes, nose, mouth and ear areas) have been selected, it can distinguish and set a plurality of selected regions corresponding to the selected face region areas, and can set different protection strengths for each of the plurality of face region areas selected through the first screen (701). For example, the electronic device can set the protection strength for the eyes to high and the protection strength for the nose to medium. According to one embodiment, when the protection strength is set differently for each face region, the electronic device can distinguish selection regions for each face region when performing face segmentation, and the protection image generation module (350) can calculate noise values ​​with different protection strengths according to the protection strengths pre-set for each distinguished selection region, reflect the noise values ​​calculated differently for each selection region, and generate a feature vector map that reflects the initialization value for the remaining area of ​​the first image (e.g., second region) excluding the selection regions.

[0123] FIG. 8 is a block diagram of a generative artificial intelligence system according to one embodiment.

[0124] Referring to FIG. 8, a generative artificial intelligence (AI) system (800) may include a user interface (810), an artificial intelligence (AI) framework (820), a generative AI model (830), a knowledge repository (840), and an application / service module (850). These components may be operated on one or more of an electronic device (101), an external electronic device (102 or 104), or a server (108). For example, the user interface (810) and the AI ​​framework (820) may be operated on the electronic device (101), and the knowledge repository (840) and the generative AI model (830) may be operated on the server (108).

[0125] According to one embodiment, a user interface (810) may receive user input (e.g., user query). User input may be received in the form of text, images, voice (e.g., natural language), video, menu selection, or a combination thereof. The user interface (810) may include various context information (e.g., running application or user location) related to the generative artificial intelligence system (800) at the time the user input is received, in addition to or instead of the user input. The user interface (810) may provide the user input or the context information to the AI ​​framework (820) and provide the result of processing therefrom to the user, for example, through the AI ​​framework (820). According to one embodiment, in addition to user input, the electronic device may provide context information obtained using information included on the screen to the AI ​​framework (820). The result may be provided in the form of text, images, voice, video, an action requested by the user (e.g., execution of a specified function or app), or a combination thereof.

[0126] According to one embodiment, the AI ​​framework (820) can identify (e.g., estimate) a user intent based on at least some of the user input or context information received from the user interface (810), control each of the relevant modules (e.g., 821, 823, or 825) to perform a function or action corresponding to the identified user intent, and coordinate collaboration between two or more modules. The AI ​​framework (820) may include a Prompt Design Module (821), an API / Plug-in Management Module (823), and an Output Modification Module (825), as illustrated in FIG. 8.

[0127] According to one embodiment, the prompt design module (821) can generate a prompt to be input to a generative AI model (830) based at least partially on user input or context information received from a user interface (810). For example, the prompt design module (821) can generate a prompt using user preferences, a prompt library, or prompt examples stored in a knowledge repository (840) based at least partially on user input or context information.

[0128] According to one embodiment, the API / plugin management module (823) may communicate, for example, via an API, with various resources (e.g., Knowledge Repository (840)) that provide said additional information when there is a request for said additional information in relation to user input. Additionally or alternatively, when a specified action (e.g., function, app, or service) is performed in response to said user input, the API / plugin management module (823) may request the application / service module (850) to perform said specified action via a corresponding API. The API / plugin management module (823) may provide information obtained from the knowledge repository (840), the application / service module (850), or another external resource to the prompt design module (821). The obtained information may be used by the prompt design module (821) to generate a prompt together with the user input, or provided to a generative AI model (830).

[0129] According to one embodiment, the output modification module (825) can fine-tune the results obtained through the generative AI model (830) as at least part of the response to user input (e.g., user query). For example, the output modification module (825) can determine whether the content of the response obtained through the generative AI model (830) is appropriate as a response to a request made by the user input. For example, the output modification module (825) can determine the degree of relevance, degree of bias (e.g., political or social bias), or degree of harmfulness (e.g., sexual or profanity) of the difference between the response obtained through the generative AI model (830) and the user input. Additionally or generally, the output modification module (825) can request that additional AI processing be performed on the obtained response, or provide the user with a hint to avoid unwanted output. For example, additional prompts can be generated through the prompt design module (821) to obtain a response again through the generative AI model (830).

[0130] According to one embodiment, the generative AI model (830) may form at least part of an artificial intelligence neural network and may include a model that generates images or a model that generates language. The image generation model may include, for example, a generative adversarial network (GAN), a variational autoencoder (VAE), or a diffusion-based model using a VAE and a Transformer. The language generation model may include, for example, a large language model (LLM), a large multimodal model (LMM), a large vision model (LVM), or a large action model (LAM). The LAM may automatically generate actions for an environment (e.g., a robot, a car, an electronic device (101), or a program (140)). Additionally, for at least some AI models (e.g., LLM), there may be a low-rank adaptation (LoRA) adaptor that is fine-tuned for, for example, a specific task or a specific situation.

[0131] FIG. 9 is a block diagram of an AI framework according to one embodiment.

[0132] FIG. 9 illustrates an AI framework (820) having on-device AI processing capabilities according to one embodiment. In this case, the AI ​​framework (820) may generate and learn a response to the user input using resources within the device, instead of sending the user input received through a user interface (810) operating on the same device (e.g., electronic device (101)) to a generative AI model (830) operating on an external device (e.g., server (108)), or additionally. Referring to FIG. 9, the AI ​​framework (820) may include a Cross-Application Action Module (910), a Personal Data Managing Module (930), an On-device AI Model (950), and an Orchestration Module (97O).

[0133] According to one embodiment, the cross-application action module (910) determines one or more additional applications required for the operation of an executed application (e.g., an assistant app) and may connect or suggest operations between the app and at least one additional application, or between a plurality of additional applications. For example, the cross-application action module (910) may execute one or more additional applications to be used to respond to a user request through the assistant app sequentially or at least partially and simultaneously. Additionally, the cross-application action module (910) may communicate with the additional applications so that the result of the execution of one additional application (e.g., content) can be shared with other additional applications.

[0134] According to one embodiment, a Personal Data Managing Module (930) may provide personal information (e.g., schedule, contact, or message information) about a user of the application (e.g., assistant app) or the additional application running on the device (e.g., electronic device 101) or other related individuals (e.g., family or friends) to another module of the AI ​​framework (820) or a related module (e.g., Generative AI Model 830) running on another device.

[0135] According to one embodiment, an on-device AI model (950) may include at least one model among one or more AI models (e.g., GAN, VAE, LLM, LMM, LVM, or LAM) operated on an external device (e.g., server (108)) or a corresponding lightweight AI model. Additionally, for said model or said lightweight model, there may be, for example, a LoRA adaptor.

[0136] According to one embodiment, an orchestration module (970) may select one or more AI models to be used to obtain a response to user input (e.g., user query). For example, the orchestration module (970) may select one or more AI models, such as an on-device AI model (950), an AI model operating on an external device (e.g., server (108)) (e.g., Generative AI Model (830)), or a third AI model (not shown) operating on another external device. When multiple AI models are selected, the orchestration module (970) may communicate with the selected models or devices so that the operation between the selected AI models and the processing of the results thereof can be coordinated between the relevant models or devices.

[0137] According to one embodiment, two or more modules of a generative AI system (800) (e.g., a cross-application action module (910) and an orchestration module (970)) may be implemented as a single module to maintain the same functionality. Various variations are possible.

[0139] According to one embodiment, a method of operation in an electronic device (e.g., the electronic device (101) of FIG. 1 or the electronic device (201) of FIG. 2) may include an operation of checking a first image among a plurality of images stored in the memory of the electronic device.

[0140] According to one embodiment, the method may include an operation of identifying a first region corresponding to a selected region in a protected target identified in the first image.

[0141] According to one embodiment, the method may include an operation of distinguishing the remaining second region, excluding the selected region from the first region and the first image.

[0142] According to one embodiment, the method may include the operation of obtaining a feature vector map in which the first region is set to a specified noise feature vector and the second region is set to an initial value.

[0143] According to one embodiment, the method may include the operation of acquiring a second image to which the feature vector map is applied as a protection image for protecting the first image.

[0144] According to one embodiment, the operation of acquiring the first image as a protective image for protecting the first image may include the operation of acquiring the second image as the protective image based on identifying that the designated noise feature vector set in the first area matches the target value of the first area.

[0145] According to one embodiment, the operation of acquiring a protection image for protecting the first image may include: an operation of updating the specified feature vector until it matches the target value based on identifying that the specified noise feature vector set in the first area does not match the target value of the first area; an operation of updating the feature vector map by setting the first area to the updated noise feature vector and setting the second area to an initial value; and an operation of updating the second image by reflecting the updated feature vector map.

[0146] According to one embodiment, the method may further include an operation of checking a first loss value to check the similarity between a noise feature vector extracted from the second image and a feature vector of the first image, an operation of checking a second loss value to check the quality loss between the first image and the second image, and an operation of updating the specified noise feature vector set in the first region with a noise specific vector generated based on the first loss value and the second loss value.

[0147] According to one embodiment, the target value of the first region may be a target noise feature vector that reflects a preset noise value for updating the specified noise feature vector based on a minimum value at which the similarity between the first image and the second image is reduced and a minimum value at which the loss of image quality is reduced.

[0148] According to one embodiment, the method may further include the operation of obtaining a total loss value as a multiplied sum between the first loss value and the second loss value, and the operation of generating the noise value for specifying the specified noise feature vector based on the total loss value.

[0149] According to one embodiment, the method may further include the operation of distinguishing the first region and the second region in the first image to obtain a selection region map before obtaining the feature vector map.

[0150] According to one embodiment, the selection area map may have a maximum value of 1 set in the first area and an initial value of 0 set in the second area.

[0151] According to one embodiment, the method may further include the operation of displaying a first screen on a display of the electronic device, the first screen including objects for setting a protection strength for protecting the first image.

[0152] According to one embodiment, the method may further include an operation of setting the target value based on the protection strength of the selected object, based on the selection of one of the objects for setting the protection strength through the first screen.

[0153] According to one embodiment, in a non-transient storage medium storing one or more programs, the program may include an instruction that causes the electronic device to perform an operation of identifying a first image among a plurality of images stored in the memory of the electronic device when executed by at least one processor of the electronic device.

[0154] According to one embodiment, the program may include an instruction that causes the electronic device to perform an operation of identifying a first region corresponding to a selected region in a protected target identified in the first image when executed by at least one processor of the electronic device.

[0155] According to one embodiment, the program may include instructions that, when executed by at least one processor of an electronic device, cause the electronic device to perform an operation of distinguishing the remaining second region excluding the selected region from the first region and the first image.

[0156] According to one embodiment, the program may include an instruction that, when executed by at least one processor of an electronic device, causes the electronic device to execute an operation of acquiring a feature vector map in which the first region is set to a specified noise feature vector and the second region is set to an initial value.

[0157] According to one embodiment, the program may include instructions that, when executed by at least one processor of an electronic device, cause the electronic device to perform an operation of acquiring a second image to which the feature vector map is applied as a protection image for protecting the first image.

[0158] According to one embodiment, the operation of acquiring a protection image for protecting the first image may include the operation of acquiring the second image as the protection image based on identifying that the designated noise feature vector set in the first area matches the target value of the first area, and the operation of updating the designated feature vector until it matches the target value based on identifying that the designated noise feature vector set in the first area does not match the target value of the first area, setting the first area to the updated noise feature vector and setting the second area to an initial value to update the feature vector map, and updating the second image by reflecting the updated feature vector map.

[0159] According to one embodiment, this document specifies a selected area from a protected target and, during the process of generating noise, directly restricts the selected area to enforce the placement of noise; thereby, regarding the target (object / person) to be protected within an image, it is possible to protect the target so that a model that recognizes and manipulates the protected target cannot recognize or manipulate the target. Accordingly, this document can improve the protection effect and reduce image degradation through efficient noise by resolving the conventional problem where inefficient or unintended noise is also inserted, causing image degradation. In addition, various effects that can be identified directly or indirectly through this document may be provided. The effects obtainable from this document are not limited to those mentioned above, and other unmentioned effects will be clearly understood by those skilled in the art to which this document belongs from the description below.

[0160] Furthermore, the embodiments of this document are presented for the purpose of explaining and understanding the technical content and are not intended to limit the scope of the technology described herein. Accordingly, the scope of this document should be interpreted to include all modifications or various other embodiments based on the technical concept of this document.

[0161] The electronic device according to the various embodiments disclosed in this document may be of various forms. The electronic device may include, for example, a portable communication device (e.g., a smartphone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a consumer electronics device. The electronic device according to the embodiments of this document is not limited to the devices described above.

[0162] The various embodiments of this document and the terms used therein are not intended to limit the technical features described in this document to specific embodiments, and should be understood to include various modifications, equivalents, or substitutions of said embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of said items unless the relevant context clearly indicates otherwise. In this document, phrases such as "A or B," "at least one of A and B," "at least one of A or B," "A, B or C," "at least one of A, B and C," and "at least one of A, B, or C" may each include any one of the items listed together in the corresponding phrase, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used simply to distinguish said components from other said components and do not limit said components in any other aspect (e.g., importance or order). Where any (e.g., 1st) component is referred to as “coupled” or “connected” to another (e.g., 2nd) component, with or without the terms “functionally” or “communicationly,” it means that said any component may be connected to said other component directly (e.g., via a wire), wirelessly, or through a third component.

[0163] The term “module” as used in the various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit, for example. A module may be a component formed integrally, or a minimum unit of said component or a part thereof that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0164] Various embodiments of the present document may be implemented as software (e.g., program (140)) comprising one or more instructions stored in a storage medium (e.g., internal memory (136) or external memory (138)) readable by a machine (e.g., electronic device (101)). For example, a processor (e.g., processor (120)) of the machine (e.g., electronic device (101)) may call at least one of the one or more instructions stored in the storage medium and execute it. This enables the machine to be operated to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code that can be executed by an interpreter. The storage medium readable by the machine may be provided in the form of a non-transitory storage medium. Here, 'non-temporary' simply means that the storage medium is a tangible device and does not contain a signal (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily.

[0165] According to one embodiment, the method according to the various embodiments disclosed herein may be provided by being included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or an application store (e.g., Play Store). TM It can be distributed online (e.g., downloaded or uploaded) through ) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created on a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0166] According to various embodiments, each component (e.g., module or program) of the components described above may include a singular or multiple entities, and some of the multiple entities may be separated and placed in other components. According to various embodiments, one or more of the components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Generally or additionally, multiple components (e.g., module or program) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each of the multiple components in the same or similar manner as those performed by the corresponding component among the multiple components prior to integration. According to various embodiments, operations performed by the module, program, or other components may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

Claims

Claim 1 An electronic device (101, 201) comprises: a processor (120, 210) including a processing circuit; and a memory (130, 220) storing instructions, wherein the instructions, when executed by the processor, cause the electronic device to: identify a first image (301) among a plurality of images stored in the memory; identify a first region corresponding to a selected region in a protected target identified in the first image; distinguish the first region and the remaining second region excluding the selected region in the first image; obtain a feature vector map in which the first region is set as a designated noise feature vector and the second region is set as an initial value; and obtain a second image (302) to which the feature vector map is applied as a protected image for protecting the first image. Claim 2 An electronic device according to claim 1, wherein the instructions, when executed by the processor, cause the electronic device to acquire the second image as the protection image based on identifying that the designated noise feature vector set in the first region matches the target value of the first region. Claim 3 An electronic device according to claim 1 or 3, wherein the instructions, when executed by the processor, cause the electronic device to update the specified feature vector until the specified noise feature vector matches the target value based on identifying that the specified noise feature vector set in the first region does not match the target value of the first region, set the first region to the updated noise feature vector and set the second region to an initial value to update the feature vector map, and update the second image by reflecting the updated feature vector map. Claim 4 An electronic device according to any one of claims 1 to 3, wherein the instructions, when executed by the processor, cause the electronic device to determine a first loss value for determining the similarity between a noise feature vector extracted from the second image and a feature vector of the first image, determine a second loss value for determining the quality loss between the first image and the second image, and update the designated noise feature vector set in the first region with a noise specific vector generated based on the first loss value and the second loss value. Claim 5 An electronic device according to any one of claims 1 to 4, wherein the target value of the first region is a target noise feature vector reflecting a preset noise value for updating the specified noise feature vector based on a minimum value at which the similarity between the first image and the second image is reduced and a minimum value at which the loss of image quality is reduced. Claim 6 An electronic device according to any one of claims 1 to 5, wherein the instructions, when executed by the processor, cause the electronic device to obtain a total loss value as a multiplied sum between the first loss value and the second loss value, and to generate a noise value for specifying the specified noise feature vector based on the total loss value. Claim 7 An electronic device according to any one of claims 1 to 6, wherein, when executed by the processor, the instructions cause the electronic device to distinguish the first region and the second region in the first image and obtain a selection region map before obtaining the feature vector map, wherein the selection region map has a maximum value of 1 set in the first region and an initial value of 0 set in the second region. Claim 8 An electronic device according to any one of claims 1 to 7, wherein the instructions, when executed by the processor, cause the electronic device to display a first screen on the display (160, 230) of the electronic device, the first screen including objects for setting a protection strength for protecting the first image. Claim 9 An electronic device according to any one of claims 1 to 8, wherein the instructions, when executed by the processor, cause the electronic device to set the target value based on the protection strength of the selected object, based on identifying that one of the objects for setting the protection strength is selected through the first screen. Claim 10 A method of operation in an electronic device (101, 201), comprising: a first image (301) among a plurality of images stored in a memory (130, 220) of the electronic device; a first region corresponding to a selected region in a protected target identified in the first image; a second region excluding the selected region in the first image and the first region; a feature vector map in which the first region is set as a designated noise feature vector and the second region is set as an initial value; and a second image (302) to which the feature vector map is applied is obtained as a protected image for protecting the first image. Claim 11 A method according to claim 10, wherein the operation of acquiring the first image as a protective image for protecting the first image includes the operation of acquiring the second image as the protective image based on identifying that the designated noise feature vector set in the first area matches the target value of the first area. Claim 12 A method according to claim 10 or 11, wherein the operation of acquiring a protection image for protecting the first image comprises: an operation of updating the specified feature vector until the specified noise feature vector matches the target value based on identifying that the specified noise feature vector set in the first area does not match the target value of the first area; an operation of updating the feature vector map by setting the first area to the updated noise feature vector and setting the second area to an initial value; and an operation of updating the second image by reflecting the updated feature vector map. Claim 13 A method according to any one of claims 10 to 12, wherein the method further comprises: a first loss value for determining the similarity between a noise feature vector extracted from the second image and a feature vector of the first image; a second loss value for determining the quality loss between the first image and the second image; and an operation of updating the designated noise feature vector set in the first region with a noise specific vector generated based on the first loss value and the second loss value. Claim 14 A method according to any one of claims 10 to 13, wherein the target value of the first region is a target noise feature vector reflecting a preset noise value for updating the specified noise feature vector based on a minimum value at which the similarity between the first image and the second image is reduced and a minimum value at which the loss of image quality is reduced. Claim 15 A method according to any one of claims 10 to 14, wherein the method further comprises: an operation of obtaining a total loss value as a multiplied sum between the first loss value and the second loss value; and an operation of generating a noise value for specifying the specified noise feature vector based on the total loss value. Claim 16 In any one of claims 10 to 15, the method further comprises the operation of distinguishing the first region and the second region in the first image to obtain a selection region map before obtaining the feature vector map, wherein the selection region map has a maximum value of 1 set in the first region and an initial value of 0 set in the second region. Claim 17 A method according to any one of claims 10 to 16, wherein the method further comprises the operation of displaying a first screen containing objects for setting a protection strength for protecting the first image on a display (160, 230) of the electronic device. Claim 18 A method according to any one of claims 10 to 16, wherein the method further comprises the operation of setting the target value based on the protection strength of the selected object, based on the selection of one of the objects for setting the protection strength through the first screen. Claim 19 A non-transient storage medium for storing one or more programs, wherein the program comprises instructions that, when executed by at least one processor (120, 210) of an electronic device (101, 201), cause the electronic device to perform: an operation of identifying a first image (302) among a plurality of images stored in the memory (130, 220) of the electronic device; an operation of identifying a first region corresponding to a selected region in a protected target identified in the first image; an operation of distinguishing the first region from the remaining second region excluding the selected region in the first image; an operation of obtaining a feature vector map in which the first region is set as a designated noise feature vector and the second region is set as an initial value; and an operation of obtaining a second image (302) to which the feature vector map is applied as a protected image for protecting the first image. Claim 20 A non-transient storage medium according to claim 19, wherein the operation of acquiring a protection image for protecting the first image comprises: an operation of acquiring the second image as the protection image based on identifying that the designated noise feature vector set in the first area matches the target value of the first area; and an operation of updating the designated feature vector until it matches the target value based on identifying that the designated noise feature vector set in the first area does not match the target value of the first area, setting the first area to the updated noise feature vector and setting the second area to an initial value to update the feature vector map, and updating the second image by reflecting the updated feature vector map.

Images