Electronic device for displaying content, operating method thereof, and recording medium

Abstract

An electronic device for displaying content and an operating method thereof are disclosed. The electronic device according to one embodiment may comprise one or more processors and a memory for storing instructions. When executed by the one or more processors, the instructions can instruct the electronic device to perform the operations of: receiving a display command for content; using metadata of the content from which the display command is received, so as to identify whether the content is edited through a generative AI model; and, when it is identified that the content is edited through the generative AI model, displaying, together with the content, an indicator indicating that the content has been edited through the generative AI model according to whether an edited area of the content is displayed.

Description

Electronic device for displaying content, method of operation thereof, and recording medium

[0001] Embodiments of the present disclosure relate to an electronic device for displaying content, a method of operating the same, and a recording medium.

[0002] A variety of content can be created or edited through electronic devices. For example, technologies have been developed to generate text-based content using generative AI models, or to modify existing content. For example, generative AI models are providing various services, such as expanding the size of content, replacing specific areas within content with different objects, or redrawing the entire content to change its overall mood.

[0003] The above information may be provided as background art to aid in understanding the present disclosure. No claim or determination is made as to whether any of the above-described matters constitute prior art related to the present disclosure.

[0004] An electronic device according to one embodiment may include a memory including one or more processors and one or more storage media storing instructions. When the instructions are executed by the one or more processors, the instructions may cause the electronic device to receive a display command for content, identify whether the content is content edited through a generative AI model using metadata of the content for which the display command has been received, and, if the content is identified as content edited through the generative AI model, display an indicator indicating that the content has been edited through the generative AI model together with the content, depending on whether an editing area of ​​the content is displayed.

[0005] According to one embodiment, an operating method of an electronic device may receive a display command for content. The operating method of the electronic device may use metadata of the content for which the display command has been received to identify whether the content is content edited through a generative AI model. If the content is identified as content edited through the generative AI model, the operating method of the electronic device may display an indicator indicating that the content has been edited through the generative AI model together with the content, depending on whether an editing area of ​​the content is displayed.

[0006] According to one embodiment, instructions stored on a non-transitory computer-readable recording medium, when executed by one or more processors, can cause the electronic device to perform operations of a method of operating the electronic device.

[0007] In connection with the description of the drawings, the same or similar reference numerals may be used for the same or similar components.

[0008] FIG. 1 is a block diagram of an electronic device within a network environment according to various embodiments.

[0009] FIG. 2 is a block diagram illustrating an electronic device displaying content according to one embodiment.

[0010] FIG. 3 is a diagram illustrating a method for storing metadata of content on which AI editing has been performed according to one embodiment.

[0011] FIG. 4 is a drawing for explaining a method for displaying an indicator of content according to one embodiment.

[0012] FIG. 5 is a diagram illustrating a method for displaying content on which AI editing has been performed according to one embodiment.

[0013] FIG. 6 is a diagram for explaining a content display policy according to one embodiment.

[0014] FIG. 7 is a drawing for explaining a method of displaying outpainted edited content according to one embodiment.

[0015] FIG. 8 is a drawing for explaining a method for displaying an indicator of outpainted edited content according to one embodiment.

[0016] FIG. 9 is a drawing showing a display method according to screen manipulation of content on which outpainting editing has been performed according to one embodiment.

[0017] FIG. 10 is a drawing showing a display method according to screen manipulation of content on which outpainting editing has been performed according to one embodiment.

[0018] Fig. 11 is a drawing showing a display method according to screen manipulation of content on which outpainting editing has been performed according to one embodiment.

[0019] Fig. 12 is a drawing showing a method for generating a thumbnail image of content according to one embodiment.

[0020] FIG. 13 is a diagram illustrating a method for displaying an indicator of content on which outpainting editing has been performed according to one embodiment.

[0021] FIG. 14 is a diagram illustrating a method for displaying an indicator of content on which inpainting editing has been performed according to one embodiment.

[0022] Fig. 15 is a drawing showing a method for displaying indicators of multimedia content according to one embodiment.

[0023] Fig. 16 is a diagram illustrating a method for displaying an indicator of text content according to one embodiment.

[0024] FIG. 17 is an example of a generative artificial intelligence system according to one embodiment.

[0025] Hereinafter, embodiments will be described in detail with reference to the attached drawings. However, the present disclosure may be implemented in various different forms and is not limited to the embodiments described herein. In the description with reference to the attached drawings, identical components will be assigned the same reference numerals regardless of the drawing numbers, and redundant descriptions thereof will be omitted.

[0026] FIG. 1 is a block diagram of an electronic device (101) within a network environment (100) according to various embodiments. Referring to FIG. 1, in the network environment (100), the electronic device (101) may communicate with an electronic device (102) via a first network (198) (e.g., a short-range wireless communication network), or may communicate with at least one of an electronic device (104) or a server (108) via a second network (199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (101) may communicate with the electronic device (104) via the server (108). According to one embodiment, the electronic device (101) may include a processor (120), a memory (130), an input module (150), an audio output module (155), a display module (160), an audio module (170), a sensor module (176), an interface (177), a connection terminal (178), a haptic module (179), a camera module (180), a power management module (188), a battery (189), a communication module (190), a subscriber identification module (196), or an antenna module (197). In some embodiments, the electronic device (101) may omit at least one of these components (e.g., the connection terminal (178)), or may have one or more other components added. In some embodiments, some of these components (e.g., the sensor module (176), the camera module (180), or the antenna module (197)) may be integrated into one component (e.g., the display module (160)).

[0027] The processor (120) may, for example, execute software (e.g., a program (140)) to control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) and perform various data processing or operations. According to one embodiment, as at least a part of the data processing or operations, the processor (120) may store commands or data received from other components (e.g., a sensor module (176) or a communication module (190)) in a volatile memory (132), process the commands or data stored in the volatile memory (132), and store result data in a non-volatile memory (134). The processor (120) may also be implemented as a system on chip (SoC) or an integrated circuit (IC) that performs processing. The processor (120) may include one or more processors, and the operations of the electronic device (101) described in the present disclosure may be performed by a single processor or by a combination of multiple processors. When the operations of the electronic device (101) are performed by a combination of multiple processors, any one processor included in the combination of processors may perform some of the operations of the electronic device (101). For example, the processor (120) may correspond to multiple processors that collectively perform a plurality of operations by dividing them among the processors.

[0028] According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or an auxiliary processor (123) (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor) that can operate independently or together with the main processor (121). For example, when the electronic device (101) includes the main processor (121) and the auxiliary processor (123), the auxiliary processor (123) may be configured to use less power than the main processor (121) or to be specialized for a given function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as a part thereof.

[0029] The auxiliary processor (123) may control at least a portion of functions or states associated with at least one component (e.g., a display module (160), a sensor module (176), or a communication module (190)) of the electronic device (101), for example, on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (123) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (180) or a communication module (190)). In one embodiment, the auxiliary processor (123) (e.g., a neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. This learning can be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or can be performed through a separate server (e.g., server (108)). The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include multiple artificial neural network layers.The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to, or alternatively to, a hardware structure, an artificial intelligence model may include a software structure.

[0030] The memory (130) can store various data used by at least one component (e.g., the processor (120) or the sensor module (176)) of the electronic device (101). The data can include, for example, software (e.g., the program (140)) and input data or output data for commands related thereto. The memory (130) can include a volatile memory (132) or a non-volatile memory (134). The memory (130) can store at least one instruction executable by the processor (120). The memory (130) can include one or more memories, and instructions for controlling the processor (120) to perform operations of the electronic device (101) described in the present disclosure can be stored in one memory or can be divided and stored in multiple memories.

[0031] The program (140) may be stored as software in the memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0032] The input module (150) can receive commands or data to be used in a component of the electronic device (101) (e.g., a processor (120)) from an external source (e.g., a user) of the electronic device (101). The input module (150) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0033] The audio output module (155) can output audio signals to the outside of the electronic device (101). The audio output module (155) can include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. In one embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

[0034] The display module (160) can visually provide information to an external device (e.g., a user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector, and a control circuit for controlling the device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by the touch. The display module (160) may be implemented with an illustrative foldable structure and / or a rollable structure. For example, the size of the display screen of the display module (160) may be reduced when folded, and may be expanded when unfolded.

[0035] The audio module (170) can convert sound into an electrical signal, or vice versa, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150), output sound through the sound output module (155), or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphone) directly or wirelessly connected to the electronic device (101).

[0036] The sensor module (176) can detect the operating status (e.g., power or temperature) of the electronic device (101) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. According to one embodiment, the sensor module (176) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0037] The interface (177) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (101) with an external electronic device (e.g., the electronic device (102)). In one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0038] The connection terminal (178) may include a connector that allows the electronic device (101) to be physically connected to an external electronic device (e.g., the electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0039] A haptic module (179) can convert electrical signals into mechanical stimuli (e.g., vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic sensations. In one embodiment, the haptic module (179) can include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

[0040] The camera module (180) can capture still images and videos. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0041] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented, for example, as at least a part of a power management integrated circuit (PMIC).

[0042] A battery (189) may power at least one component of the electronic device (101). In one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0043] The communication module (190) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may operate independently from the processor (120) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., a Wi-Fi communication module, a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (194) (e.g., a local area network (LAN) communication module, or a power line communication module). Among these communication modules, the corresponding communication module can communicate with an external electronic device (104) via a first network (198) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (199) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules can be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can verify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) by using subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (196).

[0044] The wireless communication module (192) can support 5G networks and next-generation communication technologies following the 4G network, such as NR access technology (new radio access technology). The NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and connection of multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency communications)). The wireless communication module (192) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The wireless communication module (192) can support various technologies for securing performance in a high-frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), an external electronic device (e.g., the electronic device (104)), or a network system (e.g., the second network (199)). According to one embodiment, the wireless communication module (192) can support a peak data rate (e.g., 20 Gbps or more) for eMBB realization, a loss coverage (e.g., 164 dB or less) for mMTC realization, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL), or 1 ms or less for round trip) for URLLC realization.

[0045] The antenna module (197) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). In one embodiment, the antenna module (197) may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). In one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (198) or the second network (199), may be selected from the plurality of antennas by, for example, the communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device through the selected at least one antenna. In some embodiments, in addition to the radiator, another component (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module (197).

[0046] According to various embodiments, the antenna module (197) may form a mmWave antenna module. According to one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent a first side (e.g., a bottom side) of the printed circuit board and capable of supporting a designated high frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) disposed on or adjacent a second side (e.g., a top side or a side side) of the printed circuit board and capable of transmitting or receiving signals in the designated high frequency band.

[0047] At least some of the above components can be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)).

[0048] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) via a server (108) connected to a second network (199). Each of the external electronic devices (102 or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations executed in the electronic device (101) may be executed in one or more of the external electronic devices (102, 104, or 108). For example, when the electronic device (101) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (101) may, instead of or in addition to executing the function or service itself, request one or more external electronic devices to perform the function or at least a part of the service. One or more external electronic devices that receive the request may execute at least a portion of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may process the result as is or additionally and provide it as at least a portion of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used, for example. The electronic device (101) may provide an ultra-low latency service by using distributed computing or mobile edge computing, for example. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server utilizing machine learning and / or a neural network. According to one embodiment, the external electronic device (104) or the server (108) may be included in the second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0049] Electronic devices according to various embodiments disclosed in this disclosure may take various forms. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. Electronic devices according to embodiments of this disclosure are not limited to the aforementioned devices.

[0050] The various embodiments of the present disclosure and the terminology used therein are not intended to limit the technical features described in the present disclosure to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly indicates otherwise. In the present disclosure, each of the phrases "A or B," "at least one of A and B," "at least one of A or B," "A, B, or C," "at least one of A, B, and C," and "at least one of A, B, or C" can include any one of the items listed together in the corresponding phrase among the phrases, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used merely to distinguish one component from another, and do not limit the components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as "coupled" or "connected" to another component (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0051] The term "module" used in various embodiments of the present disclosure may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or part of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0052] Various embodiments of the present disclosure may be implemented as software (e.g., a program (140)) including one or more instructions stored in a storage medium (e.g., an internal memory (136) or an external memory (138)) readable by a machine (e.g., an electronic device (101) of FIG. 1). For example, a processor (e.g., a processor (120)) of a machine (e.g., an electronic device (101)) may call at least one command among the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' simply means that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently or temporarily on the storage medium.

[0053] According to one embodiment, the method according to various embodiments disclosed in the present disclosure may be provided as included in a computer program product. The computer program product may be traded as a commodity between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) via an application store (e.g., Play Store™) or directly between two user devices (e.g., smart phones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0054] According to various embodiments, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separated and arranged in other components. According to various embodiments, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to various embodiments, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

[0055]

[0056] FIG. 2 is a block diagram illustrating an electronic device displaying content according to one embodiment.

[0057] Referring to FIG. 2, the electronic device (200) may include a processor (210), a memory (220), a user interface (230), a display (240), and a generative AI model (250). However, the configuration of the electronic device (200) is merely an example and is not limited to the above example. For example, the generative AI model (250) may be included as a component of the electronic device (200), or may be included in an external server (not shown) and interact with the electronic device (200) via wired or wireless communication.

[0058] According to one embodiment, the processor (210) may receive a display command for content through the user interface (230). For example, the processor (210) may receive a display command for content selected from among a plurality of contents stored in the memory (220) or stored in an external server through the user interface (230). The content of the present disclosure may include still images, moving images, audio, text (e.g., documents), animations, and multimedia content that combines the contents.

[0059] According to one embodiment, the user interface (230) may include a graphical user interface (GUI) that interacts with a user using graphical elements. The graphical user interface may provide an intuitive and visually rich user experience through graphical elements such as icons, buttons, windows, and menus.

[0060] According to one embodiment, the user interface (230) may include a touch user interface (TUI) based on a graphical user interface. The touch user interface may provide an intuitive and natural user experience through gesture input via touch on a touch screen. The touch screen may correspond to a display (240) including a touch sensor.

[0061] According to one embodiment, the user interface (230) may include a voice user interface (VUI) that interacts via voice commands. The voice user interface may provide a user experience that allows for more convenient and faster command input through voice recognition technology.

[0062] According to one embodiment, the user interface (230) may include a gesture-based interface that recognizes the user's physical movements or gestures to enable interaction. A gesture-based interface can provide a more immersive user experience by providing more natural interactions through camera- or motion sensor-based motion recognition.

[0063] However, the type of user interface (230) is only one example and is not limited to the above example.

[0064] According to one embodiment, the processor (210) can display content on the display (240) according to a received display command, and can perform editing on the content displayed on the display (240) according to an additional received editing command through the user interface (230).

[0065] According to one embodiment, the processor (210) may output audio or audio content included in the content through a speaker according to a received display command, and may also perform editing on the content output through the speaker according to an additional editing command received through the user interface (230).

[0066] In one embodiment, the content may be output through device form factors such as desktops, laptops, tablets, smartphones, wearable devices, extended reality (XR) devices, foldables, and rollables.

[0067] According to one embodiment, the device form factor may include a display (240) such as a liquid crystal display (LCD), a light emitting diode (LED), a quantum dot light emitting diode (QLED), an organic light emitting diode (OLED), a flexible display, and a 3D display.

[0068] In one embodiment, the processor (210) may perform general editing or AI editing using a generative AI model on content based on a received editing command. More specifically, general editing may refer to editing that does not utilize a generative AI model, such as simply adding characters entered through the user interface (230), adding objects such as existing stickers, or changing the color of certain objects in the content.

[0069] AI editing may refer to editing using a generative AI model (250), such as changing the overall style of the content, creating and displaying additional objects in a certain area of ​​the content, or creating and adding and expanding a new area that did not previously exist. However, the types of AI editing are not limited to the examples above, and may include various editing methods, including changing, deleting, moving, enlarging, and reducing existing objects displayed in the content using the generative AI model (250).

[0070] According to one embodiment, the processor (210) may manage metadata to update information about content on which general editing or AI editing has been performed. More specifically, when general editing has been performed on content, the processor (210) may add, replace, and / or modify metadata with information about the content changed as a result of the general editing.

[0071] According to one embodiment, when AI editing is performed on content, the processor (210) may distinguish and store information about the edited area identified as a result of the AI ​​editing and the original area of ​​the content in metadata. Accordingly, when the AI-edited content is later displayed on the display (240), the processor (210) may distinguish and display the original area and the edited area through the stored metadata. Furthermore, the processor (210) may identify whether the content has been AI-edited through the stored metadata, and may display an indicator indicating that the content has been edited using a generative AI model based on the identification result.

[0072] According to one embodiment, the electronic device (200) may include a memory (220) including one or more processors (210) and one or more storage media storing instructions. The instructions, when individually or collectively executed by the one or more processors (210), may cause the electronic device (200) to receive a display command for content, identify whether the content is content edited through a generative AI model using metadata of the content for which the display command has been received, and, if the content is identified as content edited through the generative AI model, display an indicator indicating that the content has been edited through the generative AI model together with the content, depending on whether an editing area of ​​the content is displayed.

[0073] According to one embodiment, the instructions, when executed individually or collectively by one or more processors (210), may cause an indicator to be displayed along with the content when an editable area of ​​content edited through the generative AI model is displayed on the display, and may cause the indicator to not be displayed when an editable area of ​​content edited through the generative AI model is not displayed on the display.

[0074] According to one embodiment, the instructions, when executed individually or collectively by one or more processors (210), may cause an indicator to be displayed when a graphical element, such as a cursor, is superimposed over at least a portion of the edit area, or when a touch input or a stylus input is detected, in a situation where an edit area of ​​content is displayed on a display without displaying an indicator.

[0075] According to one embodiment, the instructions, when executed individually or collectively by one or more processors (210), may cause an indicator to be displayed when user input is received that causes at least a portion of an edit area to be displayed on the display, in a situation where an original area of ​​content is displayed on the display.

[0076] According to one embodiment, the instructions, when executed individually or collectively by one or more processors (210), may identify a preset display policy for each edit type for edited content through a generative AI model, and, if an edited area of ​​the content is displayed according to the identified display policy, display an indicator together with the content.

[0077] According to one embodiment, the commands, when individually or collectively executed by one or more processors (210), may cause an icon to be provided as an indicator in an area of ​​the display indicating that content has been edited by a generative AI model.

[0078] According to one embodiment, the instructions, when executed individually or collectively by one or more processors (210), may cause an indicator to be provided of a graphic element overlapping an editable area of ​​content or a graphic element highlighting a boundary of an editable area of ​​content.

[0079] According to one embodiment, the instructions, when individually or collectively executed by one or more processors (210), may cause an indicator to be displayed during a playback section of multimedia content corresponding to an editing area, if the content is multimedia content.

[0080] According to one embodiment, the indicator displayed in the multimedia content may be in the form of a progress bar.

[0081] In one embodiment, metadata may be stored separately for information about editable areas and original areas of content.

[0082] In one embodiment, the metadata may include information related to a prompt inputted into the generative AI model to generate an editable region of the content.

[0083]

[0084] FIG. 3 is a diagram illustrating a method for storing metadata of content on which AI editing has been performed, according to one embodiment. In one embodiment, at least one of the operations in FIG. 3 may be performed simultaneously or in parallel with other operations, and the order of the operations may be changed. In addition, at least one of the operations may be omitted, and other operations may be additionally performed. The operations illustrated in FIG. 3 may be performed by a processor (e.g., processor 120 of FIG. 1, processor 210 of FIG. 2) of an electronic device (e.g., electronic device 101 of FIG. 1, electronic device 200 of FIG. 2).

[0085] In operation (310), the processor may display content on a display (e.g., display (240) of FIG. 2) according to a display command for content received through a user interface (e.g., user interface (230) of FIG. 2). For example, the processor may display at least one content selected through the user interface from among a plurality of contents stored in a memory of the electronic device (e.g., memory (130) of FIG. 1, memory (220) of FIG. 2) or stored in an external server, on the display.

[0086] In one embodiment, content selected through the user interface and displayed on the display may be content that has undergone AI editing via a generative AI model or content that has not undergone AI editing.

[0087] In operation (320), the processor may perform AI editing on content displayed on the display according to an editing command received via the user interface. More specifically, the processor may perform AI editing on the content by providing information about the type of editing received via the user interface as a prompt to the generative AI model. When re-performing AI editing on content already subject to AI editing, the processor may perform AI editing using the same generative AI model or may perform AI editing using different generative AI models.

[0088] In one embodiment, if information regarding the type of edit for content received through the user interface indicates an outpainting edit, the processor may input a prompt related to the outpainting edit into the generative AI model along with the content, thereby obtaining content in which a new, non-existent region is added and expanded in addition to the original region. For example, the resolution of the content subjected to the outpainting edit may be higher than that of the original content. Alternatively, the size of the content subjected to the outpainting edit may be larger than that of the original content.

[0089] In one embodiment, if information regarding the type of edit for content received through the user interface indicates inpainting editing, the processor may input a prompt related to inpainting editing into the generative AI model along with the content, thereby obtaining content in which existing objects within the original region are modified or deleted, or new objects are added. For example, the resolution and size of the content subjected to inpainting editing may be identical to the resolution and size of the original content, but at least one or more objects within the original region may be edited (created, changed, deleted, moved, enlarged, reduced, etc.).

[0090] In one embodiment, if information regarding the type of edit received for content via the user interface relates to a new creation, the processor may input a prompt related to the new creation into the generative AI model along with the content, thereby obtaining new content that is completely different from the original content. For example, the newly created and edited content may result in a change in the style of the original area, as well as in all objects existing in the original area being edited into different objects, or in the creation and addition of new objects.

[0091] In operation (330), the processor can identify an editing area of ​​content acquired through AI editing. More specifically, when storing content acquired through AI editing, the processor can distinguish the editing area according to the type of editing of the content. For example, if the content acquired through AI editing is content that has undergone outpainting editing, the processor can identify a new area added and expanded through a generative AI model in addition to the original area as the editing area. In this case, the processor can obtain size and location information of the identified editing area.

[0092] In one embodiment, if the content acquired through AI editing is content that has undergone inpainting editing, the processor can identify the region of the existing object edited through the generative AI model and the region of the newly added object as the edit region among the existing objects within the original region. At this time, the processor can obtain size and position information about the identified edit region.

[0093] In one embodiment, if the content acquired through AI editing is content that has undergone new generation editing, the processor can identify the region of all edited objects as an editing region using the generative AI model. At this time, the processor can obtain size and location information about the identified editing region.

[0094] In operation (340), the processor may update metadata based on the edited region identified in the content acquired through AI editing. For example, the processor may distinguish information about the edited region identified in the AI-edited content and the original region based on the edit type and store the information in metadata.

[0095] According to one embodiment, if the AI-edited content is content on which outpainting editing has been performed, the processor may store in metadata the size parameters and position parameters of the original area of ​​the AI-edited content and the size parameters and position parameters of an additional and extended edit area other than the original area.

[0096] In one embodiment, the processor may additionally store in the metadata boundary parameters of the original region and the edited region, and size parameters of the entire region encompassing both the original region and the edited region. Furthermore, the processor may additionally store in the metadata a type parameter of an indicator indicating that the content subjected to outpainting editing was edited using a generative AI model, and a display area parameter in which the indicator is displayed.

[0097] The indicator of the present disclosure may correspond to a watermark comprising at least one of text, audio, still images, or video. The watermark may be displayed on the content or a thumbnail of the content, or may include an icon displayed around the content or a thumbnail of the content. The watermark may include a graphic element that overlaps a specific object corresponding to the editing area, or a graphic element that emphasizes the border of a specific object corresponding to the editing area.

[0098] According to one embodiment, if the AI-edited content is content on which inpainting editing has been performed, the processor may distinguish and store in metadata the size parameters and position parameters of the original region of the AI-edited content and the size parameters and position parameters of the edited region in which an object is edited within the original region. At this time, the processor may additionally store in the metadata boundary parameters of the original region and the edited region and user input parameters (e.g., prompts) provided to the generative AI model for the edited region. At this time, if there are multiple edit regions within the original region, the processor may distinguish and store in the metadata the edit parameters (e.g., size parameters, position parameters, boundary parameters, user input parameters, etc.) for each of the multiple edit regions.

[0099] Additionally, the processor may additionally store in the metadata a parameter of a type of indicator indicating that the content on which inpainting editing was performed was edited using a generative AI model and a parameter of a display area in which the indicator is displayed.

[0100] According to one embodiment, if the edited area of ​​the content on which inpainting editing has been performed occupies an area greater than a certain standard compared to the original area, or if the correlation between objects in the edited area and objects in the original area differs greater than a certain standard, the processor may add an icon indicating that the content on which inpainting editing has been performed has been edited using a generative AI model to the original area and store the icon, and may store information about such an icon in metadata. Alternatively, the processor may manage information about the icon of the content on which inpainting editing has been performed separately from the content, and if a display command for the content is received later, the processor may display the icon in a designated area of ​​the display based on information about the separately managed icon.

[0101]

[0102] FIG. 4 is a diagram illustrating a method for displaying indicators of content according to one embodiment. In one embodiment, at least one of the operations in FIG. 4 may be performed simultaneously or in parallel with other operations, and the order between the operations may be changed. In addition, at least one of the operations may be omitted, and other operations may be additionally performed. The operations illustrated in FIG. 4 may be performed by a processor (e.g., processor 120 of FIG. 1, processor 210 of FIG. 2) of an electronic device (e.g., electronic device 101 of FIG. 1, electronic device 200 of FIG. 2).

[0103] In operation (410), the processor may display content on a display (e.g., display (240) of FIG. 2) according to a display command for content received through a user interface (e.g., user interface (230) of FIG. 2).

[0104] In one embodiment, content selected through the user interface and displayed on the display may be content that has undergone AI editing via a generative AI model or content that has not undergone AI editing.

[0105] In operation (420), the processor can use metadata of the content displayed on the display to identify whether AI editing has been performed on the content. The metadata can be stored separately, including information about the original area of ​​the content and the edited area where AI editing has been performed. The processor can easily determine whether AI editing has been performed on the content using such metadata.

[0106] In operation (430), if the processor identifies the content as having undergone AI editing, it may display an indicator indicating that the content has been edited using a generative AI model. However, the processor may display the content in different ways based on a preset display policy depending on the type of edit performed on the content.

[0107] In one embodiment, even if the content has undergone AI editing, the processor may display the edited area where AI editing was performed according to a preset display policy, or may display only the original area. In this case, the processor may additionally determine whether the edited area where AI editing was performed is displayed on the screen and determine whether to display an indicator.

[0108] According to one embodiment, the operating method of the electronic device (101, 200) may receive a display command for content. The operating method of the electronic device (101, 200) may use metadata of the content for which the display command has been received to identify whether the content is content edited through a generative AI model. If the operating method of the electronic device (101, 200) identifies the content as content edited through the generative AI model, the operating method of the electronic device (101, 200) may display an indicator indicating that the content has been edited through the generative AI model together with the content, depending on whether or not an editing area of ​​the content is displayed.

[0109] According to one embodiment, the displaying action may include an action of displaying the indicator together with the content when the editing area of ​​the content edited through the generative AI model is displayed on the display, and an action of not displaying the indicator when the editing area of ​​the content edited through the generative AI model is not displayed on the display.

[0110] According to one embodiment, the displaying action may include an action of displaying the indicator when a graphic element such as a cursor is superimposed on at least a portion of the editing area, or when a touch input or a stylus input is detected, in a situation where the editing area of ​​the content is displayed on the display without displaying the indicator.

[0111] According to one embodiment, the displaying action may include an action of displaying the indicator when a user input is received that causes at least a portion of the editing area to be displayed on the display in a situation where the original area of ​​the content is displayed on the display.

[0112] In one embodiment, the displaying action may include identifying a display policy for each preset edit type for the content edited through the generative AI model. The displaying action may include displaying the indicator together with the content when the editable area of ​​the content is displayed according to the identified display policy.

[0113] In one embodiment, the displaying action may include providing an icon as an indicator in one area of ​​the display indicating that the content has been edited through the generative AI model.

[0114] According to one embodiment, the displaying action may include providing, as the indicator, a graphic element that overlaps an editable area of ​​the content or a graphic element that emphasizes a boundary of an editable area of ​​the content.

[0115] According to one embodiment, the displaying action may include an action of displaying the indicator during a playback section of the multimedia content corresponding to the editing area, if the content is multimedia content.

[0116]

[0117] FIG. 5 is a diagram illustrating a method for displaying content on which AI editing has been performed, according to one embodiment. In one embodiment, at least one of the operations in FIG. 5 may be performed simultaneously or in parallel with other operations, and the order of the operations may be changed. In addition, at least one of the operations may be omitted, and other operations may be additionally performed. The operations illustrated in FIG. 5 may be performed by a processor (e.g., processor 120 of FIG. 1 , processor 210 of FIG. 2 ) of an electronic device (e.g., electronic device 101 of FIG. 1 , electronic device 200 of FIG. 2 ).

[0118] In operation (510), the processor may display AI-edited content received through a user interface (e.g., the user interface (230) of FIG. 2) on a display (e.g., the display (240) of FIG. 2). For example, the processor may display AI-edited content selected through the user interface from among a plurality of contents stored in a memory of the electronic device (e.g., the memory (130) of FIG. 1, the memory (220) of FIG. 2)) or stored in an external server on the display.

[0119] In operation (520), the processor can identify the type of edit performed on the content subject to AI editing. More specifically, the processor can use metadata of the content subject to AI editing to identify whether the content has undergone outpainting editing, inpainting editing, or a new generation editing process that creates content completely different from the original content.

[0120] In operation (530), the processor may display content on which AI editing has been performed on the display based on a preset display policy according to the identified editing type. For example, if the content on which AI editing has been performed is the result of outpainting editing, the processor may preferentially display the entire area including the editing area through the image viewer, or may preferentially display either the original area or the editing area, based on the preset display policy. This display policy may be determined by the image viewer's own policy or based on user input regarding which area of ​​the image should be preferentially displayed.

[0121] According to one embodiment, when displaying content on which outpainting editing has been performed, as illustrated in FIG. 6, a display policy is set to prioritize displaying the original area, and the processor may parse size information and location (e.g., coordinates) information corresponding to the original area from the metadata of the content. Based on the parsed information, the processor may extract and display only the original area (710) of the content, as illustrated in FIG. 7.

[0122] In this case, the processor may display an object (e.g., an intent object) (720) in one area of ​​the display to provide at least one additional piece of information indicating that an editing area newly added and expanded by outpainting editing exists. If the intent object (720) displayed in one area of ​​the display is selected, the processor may parse size information and position information corresponding to the editing area from the metadata of the corresponding content, and additionally display the editing area (730) based on the parsed information.

[0123] In one embodiment, the processor may display an indicator along with the content indicating that the content has been edited by a generative AI model, according to a preset display policy.

[0124] For example, if a display policy is set to display an indicator corresponding to an icon as in FIG. 6, the processor may display an icon (810) indicating that the content is edited by a generative AI model in one area of ​​the display as in FIG. 8.

[0125] According to one embodiment, the icon (810) displayed in one area of ​​the display may have a different display form depending on the editing type. For example, the processor may display icons of different shapes depending on whether the editing type is outpainting editing or inpainting editing.

[0126] When displaying icons of the same shape, the processor may apply different colors to the icons or different text or graphic effects displayed on the icons depending on the type of editing. For example, if an outpainting edit was performed, the processor may write "out" within the icon, and if an inpainting edit was performed, the processor may write "in" within the icon. Alternatively, if an outpainting edit was performed, the processor may highlight the icon within the icon, and if an inpainting edit was performed, the processor may highlight the icon outside the icon. However, the shape of the icon used to distinguish the type of editing is merely an example and is not limited to the above examples.

[0127] Meanwhile, if the target content on which AI editing is performed is content that has already undergone AI editing, the target content may include a separate indicator. In one embodiment, if AI editing is performed again on the target content, the processor may additionally display an indicator corresponding to the re-performed AI editing, in addition to the existing indicator.

[0128] According to one embodiment, when the display policy is set to display an indicator corresponding to boundary highlighting, the processor may display a graphic element (820) for highlighting the boundary of the editing area, as shown in FIG. 8. According to one embodiment, when the display policy is set to display an indicator corresponding to area highlighting, the processor may also display a graphic element (830) overlapping the editing area, as shown in FIG. 8.

[0129] However, the types of such indicators are merely examples and are not limited to the examples above. Furthermore, while FIG. 8 illustrates an example where the editing area is a background object, if the editing area is a foreground object, the indicators can be displayed in the same manner for each foreground object.

[0130] Meanwhile, in situations where the editable area of ​​content is displayed on the display without displaying an indicator according to the display policy, the processor may display an indicator only when preset conditions are met. For example, the processor may display an indicator when a graphic element, such as a cursor, overlaps at least a portion of the editable area, or when touch or stylus input is detected.

[0131] In one embodiment, if the content subjected to AI editing is the result of inpainting editing, the processor may display the entire area including the edited area through an image viewer according to a preset display policy. At this time, the processor may display an indicator indicating that the content has been edited by a generative AI model along with the content, according to the preset display policy. The type and method of displaying the indicator may be the same as those provided in FIG. 8.

[0132]

[0133] FIG. 9 is a diagram illustrating a display method according to screen manipulation of content on which outpainting editing has been performed according to one embodiment. The operations illustrated in FIG. 9 may be performed by a processor (e.g., processor (120) of FIG. 1, processor (210) of FIG. 2) of an electronic device (e.g., electronic device (101) of FIG. 1, electronic device (200) of FIG. 2).

[0134] Referring to FIG. 9, the processor can display content that has undergone outpainting editing through an image viewer. For example, if a user input related to magnification is received through a user interface (e.g., the user interface (230) of FIG. 2), the processor can display the outpainted edited content by enlarging it to a specific magnification (e.g., 2x) corresponding to the received user input. In this case, the outpainted edited content displayed on the display may display only the original area, or only the edited area added and expanded through outpainting editing, depending on the received image area. Alternatively, the outpainted edited content displayed on the display may display both the original area and the edited area simultaneously. The example of FIG. 99 provides a case where only the original area enlarged by 2x is displayed, as in the drawing (910).

[0135] In a situation where the original area enlarged twice in this way is displayed on the display, when a user input (921) corresponding to a drag input or panning input (or panning and release input) function for selecting the original area and moving it in a specific direction through a user interface as in the drawing (920) is received, the processor can identify whether the original area and the editing area are displayed simultaneously as a result of performing the user input (921).

[0136] If the original area and the edit area are identified as being displayed simultaneously by the user input (921), the processor may terminate the enlargement of the corresponding content as shown in the drawing (930) and display the entire area including both the original area and the edit area at the initial magnification (1x) on the display. Alternatively, if the processor is identified as being displayed simultaneously by the user input (921), the processor may terminate the enlargement of the corresponding content and display only the original area at the initial magnification (1x) on the display.

[0137] However, the display method of such outpainted edited content is not limited to the above examples, but rather is a result of screen manipulation based on specific user input. For example, when non-enlarged content is displayed on the display, the processor may select and display either the original area or the edited area based on received user input (e.g., distance or speed of a drag or panning input), or may display the original area and the edited area simultaneously.

[0138]

[0139] FIG. 10 is a diagram illustrating a display method according to screen manipulation of content on which outpainting editing has been performed according to one embodiment. The operations illustrated in FIG. 10 may be performed by a processor (e.g., processor (120) of FIG. 1, processor (210) of FIG. 2) of an electronic device (e.g., electronic device (101) of FIG. 1, electronic device (200) of FIG. 2).

[0140] Referring to FIG. 10, the processor can display content on which outpainting editing has been performed via an image viewer. For example, the processor can display an original area of ​​the content enlarged to a specific magnification (e.g., 2x) as shown in drawing (1010) via a user interface (e.g., user interface (230) of FIG. 2).

[0141] In a situation where the original area enlarged by two times is displayed on the display, when a user input (1021) corresponding to a drag input or a panning input for the original area is received through a user interface as in the drawing (1020), the processor can identify the display ratio of the original area and the edit area displayed as a result of performing the user input (1021).

[0142] If the ratio of the original area to the edit area indicated by the user input (10021) is determined to be less than a preset standard, the processor may provide a visual effect in which the original area, enlarged by two times in the opposite direction of the user input (1021), bounces back, as shown in the drawing (1030). However, the method of displaying such outpainted edited content is also not limited to the above example, but is merely a result of screen manipulation based on a specific user input.

[0143]

[0144] FIG. 11 is a diagram illustrating a display method according to screen manipulation of content on which outpainting editing has been performed according to one embodiment. The operations illustrated in FIG. 11 may be performed by a processor (e.g., processor (120) of FIG. 1, processor (210) of FIG. 2) of an electronic device (e.g., electronic device (101) of FIG. 1, electronic device (200) of FIG. 2).

[0145] Referring to FIG. 11, the processor can display content on which outpainting editing has been performed via an image viewer. For example, the processor can display an original area of ​​the content enlarged to a specific magnification (e.g., 2x) as shown in drawing (1110) via a user interface (e.g., user interface (230) of FIG. 2).

[0146] In a situation where the original area enlarged by two times is displayed on the display, when a user input (1121) corresponding to a drag input or panning input function for the original area is received through a user interface as in the drawing (1120), the processor can identify the display ratio of the original area and the edit area displayed as a result of performing the user input (1121).

[0147] If it is determined that the ratio of the original area to the edit area indicated by the user input (1121) is greater than a preset standard, the processor may display the original area and the edit area simultaneously as shown in the drawing (1130), or may select and display only the edit area.

[0148]

[0149] FIG. 12 is a diagram illustrating a method for generating a thumbnail image of content according to one embodiment. The operations illustrated in FIG. 12 may be performed by a processor (e.g., processor (120) of FIG. 1, processor (210) of FIG. 2) of an electronic device (e.g., electronic device (101) of FIG. 1, electronic device (200) of FIG. 2).

[0150] Referring to FIG. 12, in operation (1210), the processor may display content on a display (e.g., display (240) of FIG. 2) according to a display command for content received through a user interface (e.g., user interface (230) of FIG. 2). For example, the processor may display content on which AI editing has been performed, selected through the user interface, from among a plurality of contents stored in a memory of the electronic device (e.g., memory (130) of FIG. 1, memory (220) of FIG. 2) or stored in an external server.

[0151] In operation (1220), the processor may perform AI editing on the original content displayed on the display based on an editing command received through the user interface. More specifically, the processor may perform AI editing on the content by providing information about the type of editing received through the user interface as a prompt to the generative AI model.

[0152] For example, if information about the type of edit for content received through the user interface is an outpainting edit, the processor can input a prompt related to the outpainting edit into the generative AI model along with the content, thereby obtaining content in which a new region that did not previously exist is added and expanded in addition to the original region.

[0153] In one embodiment, if the information about the type of edit for content received through the user interface is an inpainting edit, the processor may input a prompt related to inpainting along with the content into the generative AI model, thereby obtaining content in which existing objects within the original area are changed, deleted, moved, enlarged, reduced, or new objects are added.

[0154] In operation (1230), the processor can perform AI editing and use metadata of the acquired content to identify whether the edit type of the content is an outpainting edit.

[0155] According to one embodiment, if the content is identified as having undergone outpainting editing, the processor may generate a thumbnail image including the outpainted editing area, as in operation (1240). Simultaneously, the processor may separately generate a thumbnail image for the original area excluding the outpainted editing area. The processor may store the two thumbnail images generated in this manner in a thumbnail area or a reserved area of ​​a metadata area provided in various formats of image files for the content. Alternatively, the processor may store the two thumbnail images generated in this manner separately from the content in a separate database.

[0156] According to one embodiment, if the content is identified as having undergone inpainting editing, the processor may generate a thumbnail image including the inpainted editing area, as in operation (1250). At this time, if there are multiple inpainted editing areas, the processor may generate thumbnail images including each of the multiple inpainted editing areas. For example, if four objects in one content are changed, deleted, moved, enlarged, reduced, or newly created, the processor may generate four thumbnail images each including the editing areas for the four objects. At the same time, the processor may separately generate a thumbnail image for the original area before the inpainting editing is performed. The processor may store the multiple thumbnail images generated in this way in a thumbnail area or a reserved area of ​​a metadata area provided in various formats of image files for the content. Alternatively, the processor may store the multiple thumbnail images generated in this way separately from the content in a separate database.

[0157] According to one embodiment, if the content is identified as having undergone both outpainting and inpainting editing, the processor may generate a thumbnail image including the outpainted editing area and a thumbnail image for the original area excluding the outpainted editing area. Simultaneously, the processor may generate a thumbnail image indicating (e.g., highlighting) the inpainted editing area for the original area. If there are multiple inpainted editing areas for the original area, thumbnail images including each of the multiple inpainted editing areas may be generated. The processor may store the plurality of thumbnail images generated in this manner in a thumbnail area or a reserved area of ​​a metadata area provided in various formats of image files for the content. Alternatively, the processor may store the plurality of thumbnail images generated in this manner separately from the content in a separate database.

[0158] Meanwhile, when the processor generates and stores a thumbnail image for the content, it may add an indicator to the thumbnail image indicating that the content has been edited using a generative AI model, depending on the display policy based on the edit type of the content.

[0159]

[0160] FIG. 13 is a diagram illustrating a method for displaying an indicator of content on which inpainting editing has been performed according to one embodiment.

[0161] Referring to FIG. 13, the processor may display inpainted edited content (1340) including at least one editing area (1311) through an image viewer, as in the drawing (1310). For example, the content (1340) displayed through the image viewer may be a still image in which a foreground object corresponding to a teddy bear is generated in the editing area (1311) through a generative AI model. According to one embodiment, when the inpainted edited content is first displayed on the screen according to a display policy, the processor may not display an indicator indicating that the content has been edited through a generative AI model, as in the drawing (1310).

[0162] In one embodiment, when a graphic element (1321), such as a cursor, is overlapped over at least a portion of an editing area (1311), as in drawing (1320), or a touch input (1331) or a stylus input (e.g., a hovering input) is detected over at least a portion of an editing area (1311), as in drawing (1330), the processor may display an indicator (1322, 1332) indicating that the content has been edited via a generative AI model.

[0163] At this time, the indicator (1322, 1332) may have the form of an icon displayed in one area of ​​the content. However, the form of such indicator (1322, 1332) is only one example and is not limited to the above example, and may have the form of a graphic element that overlaps the editing area as shown in FIG. 7, or may have the form of a graphic element that emphasizes the border of the editing area.

[0164]

[0165] FIG. 14 is a diagram illustrating a method for displaying an indicator of content on which inpainting editing has been performed according to one embodiment.

[0166] Referring to FIG. 14, the processor may display inpainted edited content including at least one editable area via an image viewer. For example, the content displayed via the image viewer may be a still image in which a foreground object corresponding to a cylinder is generated in the editable area (1412) via a generative AI model. Depending on the display policy, the processor may not display an indicator indicating that the inpainted edited content has been edited via a generative AI model when the content is first displayed on the screen.

[0167] According to one embodiment, the drawing (1410) may include both an original area (1411) and an edit area (1412). In this case, even if a graphic element (1413), such as a cursor, is overlapped on at least a portion of the original area (1411), or a touch input or stylus input is detected, the processor may not display a separate indicator.

[0168] According to one embodiment, the drawing (1420) may display an original area (1421) and an edit area (1422) simultaneously. At this time, when a graphic element (1423) such as a cursor is overlapped on at least a portion of the edit area (1422), or a touch input or stylus input is detected, the processor may display an icon (1424) indicating that the corresponding content is content edited by a generative AI model on one area of ​​the display.

[0169] According to one embodiment, the drawing (1430) may display an original area (1431) and an edit area (1432) simultaneously. At this time, when a graphic element (1433), such as a cursor, is overlapped on at least a portion of the edit area (1432), or a touch input or stylus input is detected, the processor may display a graphic element (1434) to emphasize the boundary of the edit area (1422).

[0170] According to one embodiment, the drawing (1440) may display an original area (1441) and an edit area (1442) simultaneously. At this time, when a graphic element (1443) such as a cursor is overlapped on at least a portion of the edit area (1442), or when a touch input or stylus input is detected, the processor may display a graphic element (1444) that overlaps the edit area (1442).

[0171]

[0172] Fig. 15 is a drawing showing a method for displaying indicators of multimedia content according to one embodiment.

[0173] Referring to FIG. 15, the processor may display multimedia content on a display as shown in the drawing (1510). At this time, if the processor identifies that an unedited section is being displayed (e.g., played) through a generative AI model using metadata of the multimedia content, the processor may not display a separate indicator.

[0174] According to one embodiment, when it is identified that a section edited through a generative AI model is being displayed through metadata of multimedia content, as in step (1520), the processor may display an indicator (1521) indicating that the content has been edited through the generative AI model. More specifically, the processor may display a progress bar as the indicator (1521) for the multimedia content. In this case, the processor may modify the progress bar so that the attributes of the section edited through the generative AI model are distinguished from other sections.

[0175] For example, the processor may display a section of the progress bar edited through a generative AI model by modifying attributes such as color, pattern, and size to differentiate it from other sections. Alternatively, the processor may provide text or an image adjacent to a section edited through a generative AI model to indicate that the content has been edited through a generative AI model.

[0176] According to one embodiment, the processor may display attributes of the edited area and the original area differently on the progress bar (e.g., displaying at least one of a color, size, pattern, or shape of the progress bar section corresponding to the edited area differently from the progress bar section corresponding to the original area, displaying an indicator on or adjacent to the progress bar section corresponding to the edited area), even if the edited section is not currently being played (e.g., displayed on the screen) through the generative AI model.

[0177] Additionally, the processor may display an indicator indicating that the content has been edited using a generative AI model by displaying a graphic element for emphasizing the border of the editing area (1522) or by displaying a graphic element overlapping the editing area (1522) during the time when the editing area (1522) is displayed within the playback screen of the multimedia content.

[0178] Meanwhile, if the multimedia content is audio content, the processor may add sound effects related to the sound source generation of the edited section through a generative AI model, or display an indicator related to the sound source generation on the player screen when the sound source of the edited section is played.

[0179]

[0180] Fig. 16 is a diagram illustrating a method for displaying an indicator of text content according to one embodiment.

[0181] Referring to FIG. 16, the processor may display text content, such as a document, on the display. In this case, if the processor identifies that unedited text (1610) is being displayed through a generative AI model using metadata of the text content, the processor may not display a separate indicator.

[0182] In one embodiment, when a slide button (1620) for displaying text generated by a generative AI model is activated, the processor can identify whether text (1630) edited by the generative AI model is being displayed through metadata of the text content. If it is identified that text (1630) edited by the generative AI model is being displayed, the processor can display an indicator (1640) indicating that the content has been edited by the generative AI model. For example, the processor can display the indicator (1640) by shading an area including text (1630) edited by the generative AI model.

[0183] In one embodiment, the processor may display an indicator (1640) indicating that the content has been edited using the generative AI model by modifying the properties of the edited text (1630) through the generative AI model to distinguish it from other text (1610). For example, the processor may display properties such as font type, font size, and font color of the edited text (1630) through the generative AI model to distinguish it from other text (1610).

[0184]

[0185] FIG. 17 is an example of a generative artificial intelligence system according to one embodiment.

[0186] The User Query / Response Interface (1710) can receive user input. The user input can be in the form of natural language, images, and / or videos. In addition, context information can also be transmitted when the user input is transmitted. The context information can include various additional information at the time of user input. For example, the various additional information can include information on the application that the user is currently using or information on the user's location. In addition, the user input can also be in the form of a mixture of the above-described natural language, images, sounds, and context information. In addition, the user input can also be in the form of a non-natural language such as selecting a menu, selecting an option, and / or selecting an object. The User Query / Response Interface (1710) can output the result of the generative artificial intelligence system to the user. The output can be in the form of natural language or a specific content, and can also be provided in the form of generating a prompt for an action requested by the user or for the generative artificial intelligence. The User Query Interface can output the result of the generative artificial intelligence system to the user. The output can be in the form of natural language or a specific content. It is possible and possible that it may be provided in the form of an action requested by the user.

[0187] The AI ​​framework (1720) can receive user input and coordinate and control each component necessary to perform the user's intention based on the user's query.

[0188] User input received from the user query / response interface (1710) can be transmitted to a prompt design component (1721). The prompt design component (1721) can be used to generate a prompt suitable for inputting the user input into an LLM or LMM. The prompt design component (1721) can be an AI component that uses a machine learning algorithm or a neural network to develop better prompts over time. The prompt design component (1721) can access knowledge repositories (1730) including user preference data, a prompt library, and prompt examples based on the user input to generate a prompt, and transmit the generated prompt to the LLM or LMM.

[0189] The APIs / Plugins management component (1722) can communicate with external information when there is a request for additional information when passing user input as input to a generative model. The APIs / Plugins management component (1722) can establish a channel for communicating with the outside of the AI ​​Interface through APIs, and can enable access to various data sources through the established channel. In addition, the APIs / Plugins management component (1722) can request an action through the API that ultimately performs the user input, rather than an intermediate result, if the action needs to be performed in an application or service (e.g., an application / service component (1740)). Information obtained from an external source can be used to generate a prompt in the prompt design component (1721) together with the user input, or can be passed as input to the generative model.

[0190] The output modification component (1723) can fine-tune the output from the generative model. For example, the output modification component can verify that the content generated by the LLM and / or LMM is not irrelevant, does not contain biased content, or does not contain harmful content. In addition, the output modification component can determine to what extent the output matches the user's desired result and, if necessary, perform additional processing. The output modification component can additionally configure and provide hints to the user to avoid undesired output.

[0191] A generative model (e.g., a generative AI model) (1750) can generally refer to an artificial intelligence neural network that creates new types of data based on user input information. Generative models may include models that generate images and / or models that generate language. Representative models that generate images include generative adversarial networks (GANs) and variational autoencoders (VAEs), and examples include diffusion-based generative models that use VAE and transformer structures. Language generating models are models that are trained to statistically output the most appropriate output based on input values, and representative examples include models such as CHAT-GPT 3 and CHAT-GPT 4. There are also large multimodal models (LMMs) that can recognize various types of data input, such as text, images, and voice, and generate new data corresponding to them.

[0192]

[0193] The embodiments of the present invention disclosed in this specification and drawings are merely specific examples presented to easily explain the technical contents according to the embodiments of the present invention and to help understand the embodiments of the present invention, and are not intended to limit the scope of the embodiments of the present invention. Therefore, the scope of the embodiments of the present invention should be interpreted as including all changes or modified forms derived based on the technical idea of ​​the embodiments of the present invention in addition to the embodiments disclosed herein.

Claims

1. In an electronic device (101), one or more processors (120); and A memory (130) comprising one or more storage media for storing commands. Including, The above commands, when individually or collectively executed by the one or more processors (120), cause the electronic device (101) to: Receive a display command for the content, The above display command uses the metadata of the received content to identify whether the content is edited through a generative AI model, If the content is identified as being edited through the generative AI model, an indicator indicating that the content has been edited through the generative AI model is displayed together with the content, depending on whether the edit area of ​​the content is displayed. Electronic device (101).

2. In paragraph 1, The above commands, when individually or collectively executed by the one or more processors (120), cause the electronic device (101) to: When the editing area of ​​the content edited through the generative AI model is displayed on the display, the indicator is displayed together with the content, and when the editing area of ​​the content edited through the generative AI model is not displayed on the display, the indicator is not displayed. Electronic device (101).

3. In any one of paragraphs 1 and 2, The above commands, when individually or collectively executed by the one or more processors (120), cause the electronic device (101) to: In a situation where the editing area of ​​the content is displayed on the display without displaying the indicator, the indicator is displayed when a graphic element such as a cursor is overlapped on at least a part of the editing area or a touch input or a stylus input is detected. Electronic device (101).

4. In any one of paragraphs 1 to 3, The above commands, when individually or collectively executed by the one or more processors (120), cause the electronic device (101) to: In a situation where the original area of ​​the above content is displayed on the display, if a user input is received that causes at least a part of the above editing area to be displayed on the display, the indicator is displayed. Electronic device (101).

5. In any one of paragraphs 1 to 4, The above commands, when individually or collectively executed by the one or more processors (120), cause the electronic device (101) to: Identifying preset display policies for each edit type for edited content through the above generative AI model, If the editing area of ​​the above content is displayed according to the above-identified display policy, the above indicator is displayed together with the above content. Electronic device (101).

6. In any one of paragraphs 1 to 5, The above commands, when individually or collectively executed by the one or more processors (120), cause the electronic device (101) to: Provide an icon as an indicator in one area of ​​the display indicating that the content has been edited through the generative AI model. Electronic device (101).

7. In any one of paragraphs 1 to 6, The above commands, when individually or collectively executed by the one or more processors (120), cause the electronic device (101) to: Providing a graphic element that overlaps at least partly with the editing area of ​​the content or a graphic element that represents the border of the editing area of ​​the content as the indicator, Electronic device (101).

8. In any one of paragraphs 1 to 7, The above commands, when individually or collectively executed by the one or more processors (120), cause the electronic device (101) to: If the above content is multimedia content, the indicator is displayed during the playback section of the multimedia content corresponding to the editing area. Electronic device (101).

9. In any one of paragraphs 1 to 8, The indicators displayed in the above multimedia content are: An electronic device (101) in the form of a progress bar.

10. In any one of paragraphs 1 to 9, The above metadata is, An electronic device (101) in which information about the editing area and the original area of ​​the above content are stored separately.

11. In any one of paragraphs 1 to 10, The above metadata is, An electronic device (101) including information related to a prompt input into the generative AI model to create an editing area of ​​the above content.

12. In the operating method of the electronic device (101), The action of receiving a display command for content; An operation of identifying whether the content is edited through a generative AI model using metadata of the content for which the display command has been received; and If the above content is identified as content edited through the generative AI model, an action of displaying an indicator indicating that the content has been edited through the generative AI model together with the content, depending on whether the editing area of ​​the content is displayed. A method of operation including:

13. In paragraph 12, The actions indicated above are: An operation of displaying the indicator together with the content when the editing area of ​​the content edited through the generative AI model is displayed on the display, and not displaying the indicator when the editing area of ​​the content edited through the generative AI model is not displayed on the display. A method of operation including:

14. In any one of paragraphs 12 to 13, The actions indicated above are: In a situation where the editing area of ​​the above content is displayed on the display without displaying the indicator, an action of displaying the indicator when a graphic element such as a cursor is overlapped on at least a part of the editing area or a touch input or stylus input is detected. A method of operation including:

15. In any one of paragraphs 12 to 14, The actions indicated above are: An action of displaying the indicator when a user input is received that causes at least a part of the editing area to be displayed on the display in a situation where the original area of ​​the above content is displayed on the display. A method of operation including:

Images