Electronic device, method, and non-transitory computer-readable recording medium for fine-tuning ai model on basis of user preference

Abstract

As described above, this electronic device may comprise a processor. The electronic device may comprise a memory for storing instructions. The instructions, when executed by the processor, may cause the electronic device to input a prompt generated by a prompt generation model to a generative AI model, thereby generating a plurality of images via image processing of a plurality of raw images. The instructions, when executed by the processor, may cause the electronic device to, on the basis of a user action for some images among the plurality of images, identify a preference related to configuration information of each of the some images. The instructions, when executed by the processor, may cause the electronic device to fine-tune the prompt generation model on the basis of the preference so as to generate another prompt for guiding the generative AI model to generate a plurality of other images having configuration information in which a high preference is acquired.

Description

Electronic device, method, and non-transitory computer-readable recording medium for fine-tuning an AI model based on user preferences

[0001] The following descriptions relate to electronic devices, methods, and non-transitory computer-readable recording media for fine-tuning an AI model based on user preferences.

[0002] Artificial intelligence is a field of computer engineering and information technology that studies ways to enable computers to perform human-like tasks such as thinking, learning, and self-improvement. It means enabling computers to imitate human intelligent behavior.

[0003] Artificial intelligence is intended to simulate human (or biological) neural activity, such as perception and / or inference, and can be implemented by hardware, software, or a combination of these designed to perform computations for simulating neural activity.

[0004] An electronic device as described above may include a processor. The electronic device may include a memory that stores instructions. The instructions, when executed by the processor, may cause the electronic device to generate a plurality of images through image processing of a plurality of raw images by inputting a prompt generated by a prompt generation model into a generative AI model. The instructions, when executed by the processor, may cause the electronic device to identify a preference associated with configuration information of each of the plurality of images based on a user action on some of the images. The instructions, when executed by the processor, may cause the electronic device to fine-tune the prompt generation model based on the preference so that the generative AI model generates another prompt to guide the generation of a plurality of other images having configuration information that yields a high preference.

[0005] The method performed by the electronic device as described above may include an operation of generating a plurality of images through image processing of a plurality of raw images by inputting a prompt generated by a prompt generation model into a generative AI model. The method may include an operation of identifying a preference associated with setting information of each of the plurality of images based on a user action on some of the images. The method may include an operation of fine-tuning the prompt generation model based on the preference such that the generative AI model generates another prompt to guide the generation of a plurality of other images having setting information that yields a high preference.

[0006] A non-transitory computer readable storage medium as described above may store a program including instructions. The instructions, when executed by a processor of an electronic device, may cause the electronic device to generate a plurality of images through image processing of a plurality of raw images by inputting prompts generated by a prompt generation model into a generative AI model. The instructions, when executed by the processor, may cause the electronic device to identify preferences associated with configuration information of each of the plurality of images based on a user action on some of the images. The instructions, when executed by the processor, may cause the electronic device to fine-tune the prompt generation model based on the preferences so that the generative AI model generates another prompt to guide the generation of a plurality of other images having configuration information that yields a high preference.

[0007] As described above, an electronic device may include a processor. The electronic device may include a memory that stores instructions. The instructions, when executed by the processor, may cause the electronic device to generate a plurality of images through image processing of a plurality of raw images by inputting a prompt generated by a prompt generation model into a generative AI model. The instructions, when executed by the processor, may cause the electronic device to identify a preference associated with setting information of each of the plurality of images based on a user action on some of the images. The instructions, when executed by the processor, may cause the electronic device to fine-tune the generative AI model based on the preference so that the generative AI model generates other images having the setting information that yields the high preference.

[0008] The method performed by the electronic device as described above may include an operation of generating a plurality of images through image processing of a plurality of raw images by inputting a prompt generated by a prompt generation model into a generative AI model. The method may include an operation of identifying a preference associated with setting information of each of the plurality of images based on a user action on some of the images. The method may include an operation of fine-tuning the generative AI model based on the preference such that the generative AI model generates other images of the plurality of images having the setting information for which the high preference is produced.

[0009] A non-transitory computer-readable recording medium as described above may store a program including instructions. The instructions, when executed by a processor of an electronic device, may cause the electronic device to generate a plurality of images through image processing of a plurality of raw images by inputting a prompt generated by a prompt generation model into a generative AI model. The instructions, when executed by the processor, may cause the electronic device to identify a preference associated with setting information of each of the plurality of images based on a user action on some of the images. The instructions, when executed by the processor, may cause the electronic device to fine-tune the generative AI model based on the preference so that the generative AI model generates other images having the setting information that yields the high preference.

[0010] FIG. 1 is a block diagram of an electronic device within a network environment according to various embodiments.

[0011] Figure 2 is a block diagram of an electronic device according to one embodiment.

[0012] Figure 3 illustrates an example of image processing according to one embodiment.

[0013] FIG. 4A illustrates an example of fine-tuning performed by an electronic device based on preference learning, according to one embodiment.

[0014] Figure 4b illustrates an example of preference learning according to one embodiment.

[0015] FIG. 4c illustrates an example of fine-tuning performed by an electronic device according to a preference, according to one embodiment.

[0016] FIG. 4d illustrates an example of fine-tuning performed by an electronic device according to a preference, according to one embodiment.

[0017] FIG. 5 is a flowchart illustrating the operation of an electronic device according to one embodiment.

[0018] FIG. 6A is a flowchart illustrating the operation of an electronic device according to one embodiment.

[0019] FIG. 6b is a flowchart illustrating the operation of an electronic device according to one embodiment.

[0020] FIG. 6c is a flowchart illustrating the operation of an electronic device according to one embodiment.

[0021] Figure 7 is a flowchart illustrating the operation of an electronic device according to one embodiment.

[0022] Figure 8 is a flowchart illustrating the operation of an electronic device according to one embodiment.

[0023] FIG. 1 is a block diagram of an electronic device (101) within a network environment (100) according to various embodiments.

[0024] Referring to FIG. 1, in a network environment (100), an electronic device (101) may communicate with an electronic device (102) via a first network (198) (e.g., a short-range wireless communication network), or may communicate with at least one of an electronic device (104) or a server (108) via a second network (199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (101) may communicate with the electronic device (104) via the server (108). According to one embodiment, the electronic device (101) may include a processor (120), a memory (130), an input module (150), an audio output module (155), a display module (160), an audio module (170), a sensor module (176), an interface (177), a connection terminal (178), a haptic module (179), a camera module (180), a power management module (188), a battery (189), a communication module (190), a subscriber identification module (196), or an antenna module (197). In some embodiments, the electronic device (101) may omit at least one of these components (e.g., the connection terminal (178)), or may have one or more other components added. In some embodiments, some of these components (e.g., the sensor module (176), the camera module (180), or the antenna module (197)) may be integrated into one component (e.g., the display module (160)).

[0025] The processor (120) may, for example, execute software (e.g., a program (140)) to control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) and perform various data processing or operations. According to one embodiment, as at least a part of the data processing or operations, the processor (120) may store commands or data received from other components (e.g., a sensor module (176) or a communication module (190)) in a volatile memory (132), process the commands or data stored in the volatile memory (132), and store result data in a non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or an auxiliary processor (123) (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor) that can operate independently or together with the main processor (121). For example, when the electronic device (101) includes the main processor (121) and the auxiliary processor (123), the auxiliary processor (123) may be configured to use less power than the main processor (121) or to be specialized for a given function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as a part thereof.

[0026] The auxiliary processor (123) may control at least a portion of functions or states associated with at least one component (e.g., a display module (160), a sensor module (176), or a communication module (190)) of the electronic device (101), for example, on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (123) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (180) or a communication module (190)). In one embodiment, the auxiliary processor (123) (e.g., a neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. This learning can be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or can be performed through a separate server (e.g., server (108)). The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include multiple artificial neural network layers.The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to, or alternatively to, a hardware structure, an artificial intelligence model may include a software structure.

[0027] The memory (130) can store various data used by at least one component (e.g., processor (120) or sensor module (176)) of the electronic device (101). The data can include, for example, software (e.g., program (140)) and input data or output data for commands related thereto. The memory (130) can include volatile memory (132) or non-volatile memory (134).

[0028] The program (140) may be stored as software in the memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0029] The input module (150) can receive commands or data to be used in a component of the electronic device (101) (e.g., a processor (120)) from an external source (e.g., a user) of the electronic device (101). The input module (150) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0030] The audio output module (155) can output audio signals to the outside of the electronic device (101). The audio output module (155) can include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. In one embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

[0031] The display module (160) can visually provide information to an external party (e.g., a user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by the touch.

[0032] The audio module (170) can convert sound into an electrical signal, or vice versa, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150), output sound through the sound output module (155), or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphone) directly or wirelessly connected to the electronic device (101).

[0033] The sensor module (176) can detect the operating status (e.g., power or temperature) of the electronic device (101) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. According to one embodiment, the sensor module (176) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0034] The interface (177) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (101) with an external electronic device (e.g., the electronic device (102)). In one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0035] The connection terminal (178) may include a connector through which the electronic device (101) may be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0036] The haptic module (179) can convert electrical signals into mechanical stimuli (e.g., vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic sensations. According to one embodiment, the haptic module (179) can include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

[0037] The camera module (180) can capture still images and videos. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0038] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

[0039] A battery (189) may power at least one component of the electronic device (101). In one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0040] The communication module (190) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may operate independently from the processor (120) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (194) (e.g., a local area network (LAN) communication module, or a power line communication module). Among these communication modules, the corresponding communication module can communicate with an external electronic device (104) via a first network (198) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (199) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules can be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can verify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) by using subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (196).

[0041] The wireless communication module (192) can support 5G networks and next-generation communication technologies following the 4G network, such as NR access technology (new radio access technology). The NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and connection of multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency communications)). The wireless communication module (192) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The wireless communication module (192) can support various technologies for securing performance in a high-frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), an external electronic device (e.g., the electronic device (104)), or a network system (e.g., the second network (199)). According to one embodiment, the wireless communication module (192) can support a peak data rate (e.g., 20 Gbps or more) for eMBB realization, a loss coverage (e.g., 664 dB or less) for mMTC realization, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL), or 6 ms or less for round trip) for URLLC realization.

[0042] The antenna module (197) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). In one embodiment, the antenna module (197) may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). In one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (198) or the second network (199), may be selected from the plurality of antennas, for example, by the communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device via the at least one selected antenna. In some embodiments, in addition to the radiator, another component (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module (197).

[0043] According to various embodiments, the antenna module (197) may form a mmWave antenna module. In one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent a first side (e.g., a bottom side) of the printed circuit board and capable of supporting a designated high-frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) disposed on or adjacent a second side (e.g., a top side or a side side) of the printed circuit board and capable of transmitting or receiving signals in the designated high-frequency band.

[0044] At least some of the above components can be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)).

[0045] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) via a server (108) connected to a second network (199). Each of the external electronic devices (102 or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations executed in the electronic device (101) may be executed in one or more of the external electronic devices (102, 104, or 108). For example, when the electronic device (101) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (101) may, instead of or in addition to executing the function or service itself, request one or more external electronic devices to perform the function or at least a part of the service. One or more external electronic devices that receive the request may execute at least a portion of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may process the result as is or additionally and provide it as at least a portion of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used, for example. The electronic device (101) may provide an ultra-low latency service by using distributed computing or mobile edge computing, for example. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server utilizing machine learning and / or a neural network. According to one embodiment, the external electronic device (104) or the server (108) may be included in the second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0046] FIG. 2 is a block diagram of an electronic device according to an embodiment. FIG. 3 illustrates an example of image processing according to an embodiment. FIG. 4a illustrates an example of fine-tuning performed by an electronic device according to preference learning according to an embodiment. FIG. 4b illustrates an example of preference learning according to an embodiment. FIG. 4c illustrates an example of fine-tuning performed by an electronic device according to preferences according to an embodiment. FIG. 4d illustrates an example of fine-tuning performed by an electronic device according to preferences according to an embodiment.

[0047] Figures 2 to 4d can be explained with reference to Figure 1.

[0048] Referring to FIG. 2, the electronic device (101) may include a processor (120), a memory (130), a display (260), a camera (280), and a communication circuit (290). The electronic device (101) of FIG. 2 may correspond to the electronic device (101) of FIG. 1. The processor (120) of FIG. 2 may correspond to the processor (120) of FIG. 1. The memory (130) of FIG. 2 may correspond to the memory (130) of FIG. 1. The display (260) of FIG. 2 may correspond to the display module (160) of FIG. 1. The camera (280) of FIG. 2 may correspond to the camera module (180) of FIG. 1. The communication circuit (290) of FIG. 2 may correspond to the communication module (190) of FIG. 1.

[0049] In one embodiment, the memory (130) may include a program (140). In one embodiment, the program (140) may include a prompt generation model (210), a generative AI model (240), a reinforcement learning model (270), and a data collection module (290). The program (140) of FIG. 2 may correspond to the program (140) of FIG. 1. In one embodiment, the reinforcement learning model (270) may include a preference learning model (275).

[0050] In one embodiment, the prompt generation model (210), the generative AI model (240), the reinforcement learning model (270), and the preference learning model (275) may include pre-trained AI (artificial intelligence) models. In FIG. 2, the electronic device (101) is illustrated as including the prompt generation model (210), the generative AI model (240), and the reinforcement learning model (270), but this is merely an example. According to an embodiment, at least one of the prompt generation model (210), the generative AI model (240), or the reinforcement learning model (270) may be included in an external device (e.g., the electronic device (102, 104) of FIG. 1, or the server (108)). In one embodiment, when at least one of the prompt generation model (210), the generative AI model (240), or the reinforcement learning model (270) is included in the external device, the electronic device (101) may obtain the result of the program included in the external device from the external device. For example, the electronic device (101) may include a prompt generation model (210), and the external device may include a generative AI model (240) and a reinforcement learning model (270). In this case, the electronic device (101) may obtain, from the external device, the output results of the generative AI model (240) and the reinforcement learning model (270) based on the results of the prompt generation model (210).

[0051] In one embodiment, the prompt generation model (210), the generative AI model (240), the reinforcement learning model (270), and the preference learning model (275) may include an AI model including a plurality of parameters related to a neural network having a structure based on an encoder and a decoder, such as a transformer. In one embodiment, the AI ​​model may include a bi-directional model based on learning for an encoder (e.g., bidirectional encoder representations from transformers (BERT)) or an auto-encoding model (e.g., a diffusion model). In one embodiment, the AI ​​model may include an auto-regressor model based on learning for a decoder (e.g., a generative pre-trained transformer (GPT)). In one embodiment, the AI ​​model may include a sequence-to-sequence model (e.g., stable diffusion, DALL-E 2) based on learning for an encoder and a decoder. In one embodiment, the generative AI model (240) may include, but is not limited to, a large language model (LLM) for processing natural language based on massive parameters. The AI ​​model may include parameters for driving a neural network such as a convolutional neural network (CNN), a recurrent neural network (RNN), a feedforward neural network (FNN), and / or a long short-term memory (LSTM).

[0052] In one embodiment, the data collection module (290) may identify (or collect) (or obtain) a user action with respect to an output result (e.g., an image) of the generative AI model (240). In one embodiment, the user action may include one or more user inputs with respect to the output result (e.g., an image). In one embodiment, the user action may include deleting, sharing, uploading, changing the storage location, viewing, applying image effects (or corrections), and / or text and / or audio inputs related to the output image.

[0053] Below, with reference to FIG. 3, the output image generated by the generative AI model (240) is described.

[0054] In one embodiment, the prompt generation model (210) can generate a prompt (330). In one embodiment, the prompt generation model (210) can generate a prompt (330) to guide image processing on input images (310). In one embodiment, the prompt generation model (210) can include an AI model (e.g., a stable diffusion model) that can convert the input images (310) into text. In one embodiment, the image processing can include hot pixel correction, demosaicing, color interpolation, noise reduction, lens shading correction, defective pixel correction, green imbalance correction, crosstalk compensation, or auto dark level compensation (ADLC) to remove fixed pattern noise (FPN). In one embodiment, the image processing may include edge enhancement, gamma correction, color saturation correction, or hue correction. In one embodiment, the image processing may include auto white balance (AWB), auto focus (AF), auto exposure (AE), high dynamic range (HDR), tone curve adjustment, resolution adjustment (e.g., up-scale, down-scale), or format conversion (e.g., from RAW to YUV, JPEG (joint photographic experts group)).In one embodiment, image processing may include, but is not limited to, depth map generation, 3D modeling, panorama generation, feature point extraction, or image synthesis.

[0055] In one embodiment, the prompt (330) may include data for guiding the generation of output images (320) based on the input images (310). In one embodiment, the prompt (330) may be a task instruction for the generative AI model (240) (e.g., 'generate an image with specified camera setting information and / or image effects applied'). In one embodiment, the prompt (330) may be a context input to the generative AI model (240) (e.g., information about an object included in the input image, information about the arrangement of the object, or information about the composition of the object). In one embodiment, the prompt (330) may be data for setting camera setting information, composition, and / or image effects of the output images (320) generated based on the input images (310). In one embodiment, the prompt (330) may be data including a set of words.

[0056] In one embodiment, the input images (310) input to the prompt generation model (210) may be raw images acquired through the camera (280). In one embodiment, the input images (310) input to the prompt generation model (210) may be image-processed images acquired from an external source (e.g., an electronic device (102, 103), or a server (108)) through the communication circuit (290). However, the present invention is not limited thereto. The input images (310) input to the prompt generation model (210) may be images that have been image-processed through the generative AI model (240) for raw images acquired through the camera (280).

[0057] In one embodiment, the prompt generation model (210) can generate similar prompts for images with similar compositions. In one embodiment, the prompt generation model (210) can generate similar prompts for images with similar compositions so that similar image processing can be performed on images with similar compositions. Here, the composition can be determined by the arrangement and / or shape of objects included in the image. For example, the composition can include a triangle composition, a vertical composition, a horizontal composition, a vertical-horizontal composition, a heart composition, a fan composition, or a diagonal composition.

[0058] In one embodiment, the prompt generation model (210) can generate similar prompts for similar types of images. In one embodiment, the prompt generation model (210) can generate similar prompts for similar types of images so that similar image processing can be performed on similar types of images. Here, the type of image can be determined by the type of objects included in the image. For example, the type of image can include a nature photograph, a landscape photograph, an astronomical photograph, a storm photograph, an animal photograph, a macro photograph, a flower photograph, a food photograph, a portrait photograph, an aerial photograph, or a wedding photograph.

[0059] In one embodiment, the prompt generation model (210) can generate similar prompts for images of the same object. In one embodiment, the prompt generation model (210) can generate similar prompts for images of the same object so that similar image processing is performed on the images of the same object.

[0060] In one embodiment, the prompt generation model (210) can generate prompts (330) that include prompts related to composition, prompts related to type, and / or prompts related to objects. Each of the prompts related to composition, type, and object may have different targets that require image processing. For example, a prompt related to composition may indicate a prompt related to gamma correction, color saturation correction, or chromaticity correction. For example, a prompt related to type may indicate a prompt related to auto white balance, HDR, or edge enhancement. For example, a prompt related to object may indicate a prompt related to tone curve adjustment, resolution adjustment, but is not limited thereto.

[0061] In one embodiment, the prompt generation model (210) can generate a prompt (330) for each of the input images (310). For example, the prompt generation model (210) can generate a prompt (330) for image (311), a prompt (330) for image (313), a prompt (330) for image (315), a prompt (330) for image (317), and a prompt (330) for image (319). The prompts (330) for each of the input images (310) can include prompts related to composition, prompts related to types, and / or prompts related to objects. For example, for images (311, 313, 315) of the same composition, the prompts related to composition can be similar. For example, for images (311, 313, 315, 317, 319) of the same type (e.g., pictures of animals), the prompts related to the type may be similar. For example, for images (311, 313, 315) of the same object (e.g., the same dog), the prompts related to the object may be similar.

[0062] In one embodiment, the prompt generation model (210) can pass generated prompts (330) for input images (310) to the generative AI model (240).

[0063] In one embodiment, the generative AI model (240) may generate at least one output image (320) in which image processing is performed on input images (310) based on a prompt. In one embodiment, the generative AI model (240) may generate at least one output image in which setting information of the input image is changed based on the prompt. In one embodiment, the setting information may include a plurality of setting items. In one embodiment, the setting information may include information about a focal length, a flash, an international organization for standardization (ISO) sensitivity, an aperture, a shutter speed, a rotation direction, a color temperature, and / or a color space, but is not limited thereto. In one embodiment, the setting information may indicate whether hot pixel correction, demosaicing, color interpolation, noise reduction, lens shading correction, bad pixel processing, green imbalance correction, crosstalk compensation, or auto dark level compensation is performed. In one embodiment, the setting information may indicate whether edge enhancement, gamma correction, color saturation correction, or chromaticity correction is performed. In one embodiment, the setup information may indicate whether to perform auto white balance, auto focus, auto exposure, HDR, tone curve adjustment, resolution adjustment, and / or format conversion.

[0064] In one embodiment, the generative AI model (240) may output output images (320) on which image processing has been performed based on different prompts (330) depending on the composition, type, and / or object of the input images (310). For example, output images (321, 323, 325) on which the same image processing has been performed may be generated for input images (311, 313, 315) on which the composition, type, and object have the same composition, type, and object. For example, output images (317, 319) on which the same image processing has been performed may be generated for input images (317, 319) on which the type and object have the same composition, type, and object. However, the present invention is not limited thereto.

[0065] As described above, the electronic device (101) may not generate images that match the user's individual tastes when generating output images (320) by the pre-trained prompt generation model (210) and the generative AI model (240). Accordingly, examples of operations for training the preference learning model (275) and fine-tuning the prompt generation model (210) and / or the generative AI model (240) to generate images that match the user's individual tastes are described below.

[0066] Hereinafter, with reference to FIG. 4a, a method for fine-tuning a preference learning model (275) and a prompt generation model (210), and / or a generative AI model (240), based on a user's behavior (or action) is described.

[0067] In one embodiment, the prompt generation model (210) may generate prompts to guide image processing on input images (310). In one embodiment, the generative AI model (240) may generate at least one output image (320) on which image processing on the input images (310) is performed based on the prompts.

[0068] In one embodiment, the data collection module (290) can obtain a user action (401) for at least one output image (320). The user action (401) can include one or more user inputs for the output images (320). In one embodiment, the user action (401) can include deleting, sharing, uploading, changing the storage location of, viewing, applying an image effect (or correction), and / or text and / or audio input related to the output image.

[0069] In one embodiment, the preference learning model (275) may output preferences for output images of the generative AI model (240) using user actions (401). In one embodiment, the preference learning model (275) may output preferences for each of the output images of the generative AI model (240).

[0070] In one embodiment, the reinforcement learning model (270) may quantify the preference and / or identify a preference ranking of some images on which a user action has been performed, based on clustering and / or regression analysis. In one embodiment, clustering may be an algorithm for classifying a plurality of images into one or more preference groups. In one embodiment, one or more images included in a preference group may be evaluated as having the same and / or similar preferences.

[0071] In one embodiment, the reinforcement learning model (270) can train the prompt generation model (210) and / or the generative AI model (240) based on the preferences. In one embodiment, the reinforcement learning model (270) can fine-tune the prompt generation model (210) and / or the generative AI model (240) based on the preferences.

[0072] In one embodiment, the reinforcement learning model (270) can generate a fine-tuned prompt generation model and / or generative AI model by fine-tuning the prompt generation model (210) and / or generative AI model (240) based on the preferences.

[0073] Hereinafter, with reference to FIG. 4b, a method for learning a preference learning model (275) based on a user's behavior (or action) is described.

[0074] In one embodiment, the reinforcement learning model (270) can identify a user action (401) for a plurality of output images (320) generated by the generative AI model (240). In one embodiment, the reinforcement learning model (270) can identify a user action (401) based on one or more user inputs for the output images (320), but is not limited thereto. In one embodiment, the reinforcement learning model (270) can identify a user action (401) for images that were not generated by the generative AI model (240). In one embodiment, the user action (401) can include deleting, sharing, uploading, changing the storage location, viewing, applying an image effect (or correction) to the output image, and / or text and / or audio input related to the output image. Here, a user action related to an image effect (or correction) can include a user input for applying a filter to the image. Here, the filter can be a filter for adding a watercolor effect, brush stroke, blur, distortion, or pixelation to the image.

[0075] For example, the user action (401) may include user inputs for deleting the output image. For example, the user action (401) may include user inputs for sharing the output image. For example, the user action (401) may include user inputs for sharing the output image with another electronic device (102, 103). For example, the user action (401) may include user inputs for uploading the output image to a web page provided by the server (108). For example, the user action (401) may include user inputs for adding text and / or audio inputs on the output image. For example, the user action (401) may include user inputs for adding text and / or audio inputs to a web page to which the output image is uploaded. For example, the user action (401) may include user inputs for transmitting a message including text and / or audio to another electronic device (102, 103) with which the output image is shared. For example, the user action (401) may include user inputs for changing the storage location of the output image (or changing the folder). For example, the user action (401) may include user inputs for changing the storage location of one or more output images to the same storage location (or changing the folder). For example, the user action (401) may include user inputs for viewing the output image. For example, the user action (401) may include user inputs for applying an image effect (or correction) to the output image.

[0076] For example, referring to FIG. 4B, the reinforcement learning model (270) can identify a user action for deleting images (323, 325) among three sequentially captured output images (321, 323, 325). For example, the reinforcement learning model (270) can identify a user action for sharing an output image (327). For example, the reinforcement learning model (270) can identify a user action for entering a text input (403) for the shared output image (327). For example, the reinforcement learning model (270) can identify a user action for moving two output images (327, 329) to the same storage path (e.g., the same folder) (405). For example, the reinforcement learning model (270) can identify a user action to register two output images (327, 329) as favorites.

[0077] In one embodiment, the reinforcement learning model (270) can identify preferences associated with configuration information for each of the plurality of images based on user actions for the images.

[0078] In one embodiment, the reinforcement learning model (270) can identify the preference of some images on which a user action has been performed among multiple images of the same object. In one embodiment, the reinforcement learning model (270) can identify a preference rank among multiple images of the same object based on the user action. In one embodiment, the reinforcement learning model (270) can identify the preference of some images on which a user action has been performed as being higher or lower than the preference of the remaining images on which a user action has been performed for the same object. In one embodiment, the reinforcement learning model (270) can identify the preference of an image on which a user action to delete the image has been performed as being lower than the preference of other images on the same object (or lower than a reference preference). In one embodiment, the reinforcement learning model (270) can identify the preference of an image on which a user action of sharing, uploading, changing the storage location, and / or viewing the image has been performed as being higher than the preference of other images of the same object (or exceeding a reference preference). In one embodiment, the reinforcement learning model (270) can quantify the preference and / or identify a preference ranking of some images on which a user action has been performed based on clustering and / or regression analysis. In one embodiment, the clustering may be an algorithm for classifying a plurality of images into one or more preference groups. In one embodiment, the clustering may be a K-means algorithm, a mean shift algorithm, a Gaussian mixture model (GMM) algorithm, or a density-based spatial clustering of applications with noise (DBSCAN) algorithm. In one embodiment, the regression analysis may be an algorithm for evaluating the correlation (or preference) between each of the plurality of images and one or more preference groups.In one embodiment, one or more images included in a preference group may be evaluated as having the same and / or similar preferences, but is not limited thereto. The reinforcement learning model (270) may identify the preference associated with the configuration information of each of the output images (320) based on text and / or audio inputs associated with the output images. For example, if there is a positive evaluation in the content of the text and / or audio inputs associated with the output images, the reinforcement learning model (270) may identify the preference associated with the configuration information of each of the output images (320) as being relatively high. For example, if there is a negative evaluation in the content of the text and / or audio inputs associated with the output images, the reinforcement learning model (270) may identify the preference associated with the configuration information of each of the output images (320) as being relatively low. For example, the reinforcement learning model (270) can identify the user action for viewing the plurality of images (321, 323, 325, 327, 329). For example, the reinforcement learning model (270) can identify a preference for the plurality of images (321, 323, 325, 327, 329) based on the user action for viewing the plurality of images (321, 323, 325, 327, 329). For example, the reinforcement learning model (270) can identify a preference that is proportional to the number of user inputs (or the number of viewings) for viewing the plurality of images (321, 323, 325, 327, 329).

[0079] In one embodiment, the reinforcement learning model (270) can identify the preference of some images on which a user action has been performed among multiple images of the same composition. In one embodiment, the reinforcement learning model (270) can identify the preference of some images on which a user action has been performed among multiple images of the same type.

[0080] In one embodiment, the reinforcement learning model (270) can identify preferences for setting information for some images on which image processing has been performed based on object, composition, and type.

[0081] In one embodiment, the reinforcement learning model (270) may train the preference learning model (275) based on the plurality of images for which the preferences are identified based on the user action. In one embodiment, the reinforcement learning model (270) may train the preference learning model (275) such that the preferences for images produced by the preference learning model (275) have the same and / or similar preferences as the preferences produced by the user action. In one embodiment, the preference learning model (275) may be an AI model that outputs preferences for input images. In one embodiment, the preference learning model (275) may be trained to output preferences for output images of the generative AI model (240) that are the same and / or similar to the preferences identified by the user action.

[0082] In one embodiment, the preference learning model (275) may output preferences for the output images of the generative AI model (240). In one embodiment, preferences may be divided into preferences output by the preference learning model (275) and preferences resulting from user actions.

[0083] As described above, the electronic device (101) can identify a user's personalized preferences based on the user's activities (or actions). Furthermore, the electronic device (101) can identify a user's personalized preferences by collecting the user's activities (or actions) without the need for conventional questionnaires or preference tests (e.g., A / B testing) to obtain the user's preferences. Accordingly, the electronic device (101) can train a preference learning model (275) based on the user's preferences.

[0084] As described above, the electronic device (101) can evaluate the preference of multiple output images based on user actions for multiple output images for one raw image.

[0085] Additionally, the electronic device (101) may display a user interface (UI) requesting an evaluation of multiple output images for a single raw image through the display (260). In one embodiment, the electronic device (101) may obtain an evaluation of the multiple output images through the UI. In one embodiment, the electronic device (101) may identify a preference related to setting information of each of the multiple output images based on the evaluation of the multiple output images through the UI. In one embodiment, the UI may include a phrase querying multiple output images generated from the same raw image and a preference of the multiple output images.

[0086] In addition, the electronic device (101) may display a UI requesting an evaluation of the remaining output images without user action among the plurality of output images through the display (260). In one embodiment, the electronic device (101) may obtain an evaluation of the remaining output images through the UI. In one embodiment, the electronic device (101) may identify a preference related to the setting information of each of the remaining output images based on the evaluation of the remaining output images through the UI. In one embodiment, the UI may include a phrase querying the remaining output images generated from the same different raw images and the preference of the remaining output images. In one embodiment, the different raw images may have at least one of the same composition, type, and object.

[0087] Hereinafter, with reference to FIG. 4c, a method for learning a prompt generation model (210) using a learned preference learning model (275) is described.

[0088] In one embodiment, the prompt generation model (210) can generate a prompt. In one embodiment, the prompt generation model (210) can generate a prompt (450) to guide image processing for input images (420).

[0089] In one embodiment, the prompt generation model (210) can generate a prompt (450) for each of the input images (420). For example, the prompt generation model (210) can generate a prompt (450) for image (421), a prompt (450) for image (423), a prompt (450) for image (425), a prompt (450) for image (427), and a prompt (450) for image (429). The prompt (450) for each of the input images (420) can include a prompt related to composition, a prompt related to type, and / or a prompt related to object.

[0090] In one embodiment, the prompt generation model (210) can pass the generated prompt (450) for the input image to the generative AI model (240).

[0091] In one embodiment, the generative AI model (240) can generate output images (430) by performing image processing on input images (420) based on prompts.

[0092] In one embodiment, the preference learning model (275) can output preferences (440) for the output images (430) of the generative AI model (240). In one embodiment, the preference learning model (275) can output preferences (440) for each of the output images (430) of the generative AI model (240). For example, the preference learning model (275) can output preferences (440) for the output image (431), preferences (440) for the output image (433), preferences (440) for the output image (435), preferences (440) for the output image (437), and preferences (440) for the output image (439).

[0093] In one embodiment, the reinforcement learning model (270) can train the prompt generation model (210) based on the preferences (440). In one embodiment, the reinforcement learning model (270) can fine-tune the prompt generation model (210) based on the preferences (440). In one embodiment, the preferences (440) for the output images (430) of the preference learning model (275) can be understood as preferences for the prompts (450) of the prompt generation model (210) to guide the generation of the output images.

[0094] In one embodiment, the reinforcement learning model (270) can fine-tune the prompt generation model (210) to generate different prompts to guide the generative AI model (240) in generating output images having setting information that yields high preferences, based on the preferences (440) for the output images (430) of the preference learning model (275).

[0095] In one embodiment, the reinforcement learning model (270) can calculate a loss according to the preference (440). In one embodiment, the reinforcement learning model (270) can update the parameters of the prompt generation model (210) based on the loss. In one embodiment, the reinforcement learning model (270) can update the parameters of the prompt generation model (210) by backpropagating the loss. In one embodiment, the loss can be a negative preference (440), but is not limited thereto.

[0096] As described above, the electronic device (101) can train the prompt generation model (210) to generate an output image with a high user preference based on the preferences derived from the preference learning model (275) learned according to the user's preferences, so that the generative AI model (240) can generate an output image with a high user preference. Accordingly, the electronic device (101) can provide the user with an output image having the user's personalized preferences.

[0097] Hereinafter, with reference to FIG. 4c, a method for learning a generative AI model (240) using a learned preference learning model (275) is described.

[0098] In one embodiment, the prompt generation model (210) can generate a prompt. In one embodiment, the prompt generation model (210) can generate a prompt (450) to guide image processing for input images (420).

[0099] In one embodiment, the prompt generation model (210) can generate a prompt (450) for each of the input images (420). For example, the prompt generation model (210) can generate a prompt (450) for image (421), a prompt (450) for image (423), a prompt (450) for image (425), a prompt (450) for image (427), and a prompt (450) for image (429). The prompt (450) for each of the input images (420) can include a prompt related to composition, a prompt related to type, and / or a prompt related to object.

[0100] In one embodiment, the prompt generation model (210) can pass the generated prompt (450) for the input image to the generative AI model (240).

[0101] In one embodiment, the generative AI model (240) can generate output images (430) by performing image processing on input images (420) based on prompts.

[0102] In one embodiment, the preference learning model (275) can output preferences (440) for the output images (430) of the generative AI model (240). In one embodiment, the preference learning model (275) can output preferences (440) for each of the output images (430) of the generative AI model (240). For example, the preference learning model (275) can output preferences (440) for the output image (431), preferences (440) for the output image (433), preferences (440) for the output image (435), preferences (440) for the output image (437), and preferences (440) for the output image (439).

[0103] In one embodiment, the reinforcement learning model (270) can train the generative AI model (240) based on the preference (440). In one embodiment, the reinforcement learning model (270) can fine-tune the generative AI model (240) based on the preference (440). In one embodiment, the reinforcement learning model (270) can fine-tune the generative AI model (240) based on the preference (440) for the output images (430) of the preference learning model (275) so that the generative AI model (240) generates output images having setting information that yields high preferences.

[0104] In one embodiment, the reinforcement learning model (270) can calculate a loss according to the preference (440). In one embodiment, the reinforcement learning model (270) can update the parameters of the generative AI model (240) based on the loss. In one embodiment, the reinforcement learning model (270) can update the parameters of the generative AI model (240) by backpropagating the loss. In one embodiment, the loss can be a negative preference (440), but is not limited thereto.

[0105] As described above, the electronic device (101) can train the generative AI model (240) to generate an output image with a high user preference based on the preferences produced by the preference learning model (275) learned according to the user's preferences. Accordingly, the electronic device (101) can provide the user with an output image having the user's personalized preferences.

[0106] As described above, the electronic device (101) can identify a user's personalized preferences by collecting the user's activities (or actions) without the need for conventional questionnaires or preference tests (e.g., A / B testing) to obtain the user's preferences. Accordingly, the electronic device (101) can provide the user with an output image containing the user's personalized preferences without requiring significant effort from the user.

[0107] As described above, the electronic device (101) has been described as learning one of the prompt generation model (210) or the generative AI model (240) using the learned preference learning model (275), but this is merely an example. In one embodiment, the electronic device (101) may update the parameters of the prompt generation model (210) and / or the generative AI model (240) based on the loss according to the preference (440). In one embodiment, the electronic device (101) may update the parameters of the prompt generation model (210) and / or the generative AI model (240) by backpropagating the loss.

[0108] Hereinafter, with reference to FIG. 4d, a method for generating a fine-tuned prompt generation model (215) and / or a generative AI model (245) using a learned preference learning model (275) is described.

[0109] In one embodiment, the preference learning model (275) may output preferences (440) for output images of the generative AI model (240) using user actions (401). In one embodiment, the preference learning model (275) may output preferences (440) for each of the output images of the generative AI model (240).

[0110] In one embodiment, the reinforcement learning model (270) can identify the preference (440) of some images on which a user action (401) was performed among a plurality of images of the same object. In one embodiment, the reinforcement learning model (270) can identify a preference ranking among the plurality of images on which a user action (401) was performed for the same object based on the user action (401). In one embodiment, the reinforcement learning model (270) can quantify the preference and / or identify a preference ranking of some images on which a user action was performed based on clustering and / or regression analysis. In one embodiment, the clustering may be an algorithm for classifying a plurality of images into one or more preference groups. In one embodiment, the clustering may be a K-means algorithm, a mean shift algorithm, a GMM (Gaussian mixture model) algorithm, or a DBSCAN (density-based spatial clustering of applications with noise) algorithm. In one embodiment, regression analysis may be an algorithm for evaluating the correlation (or preference) between each of a plurality of images and one or more preference groups. In one embodiment, one or more images included in a preference group may be evaluated as having the same and / or similar preferences.

[0111] In one embodiment, the reinforcement learning model (270) can train the prompt generation model (210) and / or the generative AI model (240) based on the preferences (440). In one embodiment, the reinforcement learning model (270) can fine tune the prompt generation model (210) and / or the generative AI model (240) based on the preferences (440).

[0112] In one embodiment, the reinforcement learning model (270) can generate a fine-tuned prompt generation model (215) and / or generative AI model (245) by fine-tuning the prompt generation model (210) and / or generative AI model (240) based on the preferences (440).

[0113] Thereafter, the generation of a prompt for an input image (or an image acquired through a camera module (180) of FIG. 1) may utilize a fine-tuned prompt generation model (215). In addition, the generation of an output image for an input image (or an image acquired through a camera module (180) of FIG. 1) may utilize a fine-tuned generative AI model (245).

[0114] FIG. 5 is a flowchart illustrating the operation of an electronic device according to one embodiment.

[0115] Figure 5 can be explained with reference to Figures 1 to 4d.

[0116] Referring to FIG. 5, in operation 510, the electronic device (101) may perform image processing on an image. In one embodiment, the image may be a raw image acquired through a camera (280). In one embodiment, the image may be an image-processed image acquired from an external source (e.g., an electronic device (102, 103) or a server (108)) through a communication circuit (290). However, the present invention is not limited thereto. The image may be an image processed through a generative AI model (240) on a raw image acquired through the camera (280).

[0117] In one embodiment, the electronic device (101) may generate at least one output image (320) in which image processing is performed on input images (310) based on a prompt. In one embodiment, the electronic device (101) may generate at least one output image in which setting information of the input image is changed based on the prompt. In one embodiment, the setting information may include a plurality of setting items. In one embodiment, the setting information may include information about a focal length, a flash, an ISO sensitivity, an aperture, a shutter speed, a rotation direction, a color temperature, and / or a color space, but is not limited thereto. In one embodiment, the setting information may indicate whether hot pixel correction, demosaicing, color interpolation, noise reduction, lens shading correction, bad pixel processing, green imbalance correction, crosstalk compensation, or auto dark level compensation is performed. In one embodiment, the setting information may indicate whether edge enhancement, gamma correction, color saturation correction, or chromaticity correction is performed. In one embodiment, the setup information may indicate whether to perform auto white balance, auto focus, auto exposure, HDR, tone curve adjustment, resolution adjustment, and / or format conversion.

[0118] In one embodiment, the electronic device (101) may output output images (320) on which image processing has been performed based on different prompts (330) depending on the composition, type, and / or object of the input images (310). For example, output images (321, 323, 325) on which the same image processing has been performed may be generated for input images (311, 313, 315) on which the composition, type, and object have the same composition, type, and object. For example, output images (317, 319) on which the same image processing has been performed may be generated for input images (317, 319) on which the type and object have the same composition, type, and object. However, the present invention is not limited thereto.

[0119] Here, the prompt (330) may include data for guiding the generation of output images (320) based on the input images (310). In one embodiment, the prompt (330) may be a work instruction for the generation of at least one output image (320). In one embodiment, the prompt (330) may be data for setting camera setting information, composition, and / or image effects of the output images (320) generated based on the input images (310). In one embodiment, the prompt (330) may be data including a set of words.

[0120] In one embodiment, the electronic device (101) may generate prompts (330) including prompts related to composition, prompts related to type, and / or prompts related to objects. Each of the prompts related to composition, type, and object may have different targets that require image processing. For example, the prompts related to composition may indicate prompts related to gamma correction, color saturation correction, or chromaticity correction. For example, the prompts related to type may indicate prompts related to auto white balance, HDR, or edge enhancement. For example, the prompts related to objects may indicate prompts related to tone curve adjustment and resolution adjustment, but are not limited thereto.

[0121] In one embodiment, the electronic device (101) may generate a prompt (330) for each of the input images (310). For example, the prompt generation model (210) may generate a prompt (330) for image (311), a prompt (330) for image (313), a prompt (330) for image (315), a prompt (330) for image (317), and a prompt (330) for image (319). The prompt (330) for each of the input images (310) may include a prompt related to composition, a prompt related to type, and / or a prompt related to object. For example, for images (311, 313, and 315) of the same composition, the prompts related to composition may be similar. For example, for images (311, 313, 315, 317, 319) of the same type (e.g., pictures of animals), the prompts related to the type may be similar. For example, for images (311, 313, 315) of the same object (e.g., the same dog), the prompts related to the object may be similar.

[0122] In operation 520, the electronic device (101) may identify a user action for the processed image. In one embodiment, the electronic device (101) may identify a user action (401) for a plurality of output images (320). In one embodiment, the electronic device (101) may identify the user action (401) based on one or more user inputs for the output images (320), but is not limited thereto. In one embodiment, the electronic device (101) may identify a user action (401) for images that have not been processed by the electronic device (101). In one embodiment, the user action (401) may include deleting, sharing, uploading, changing the storage location, viewing, applying an image effect (or correction) to the output image, and / or text and / or audio input related to the output image. Here, a user action related to an image effect (or correction) may include a user input for applying a filter to the image. Here, the filter can be a filter for adding a watercolor effect, brush stroke, blur, distortion, or pixelation to the image.

[0123] For example, the user action (401) may include user inputs for deleting the output image. For example, the user action (401) may include user inputs for sharing the output image. For example, the user action (401) may include user inputs for sharing the output image with another electronic device (102, 103). For example, the user action (401) may include user inputs for uploading the output image to a web page provided by the server (102). For example, the user action (401) may include user inputs for adding text and / or audio inputs on the output image. For example, the user action (401) may include user inputs for adding text and / or audio inputs to a web page to which the output image is uploaded. For example, the user action (401) may include user inputs for transmitting a message including text and / or audio to another electronic device (102, 103) with which the output image is shared. For example, the user action (401) may include user inputs for changing the storage location of the output image (or changing the folder). For example, the user action (401) may include user inputs for changing the storage location of one or more output images to the same storage location (or changing the folder). For example, the user action (401) may include user inputs for viewing the output image. For example, the user action (401) may include user inputs for applying an image effect (or correction) to the output image.

[0124] In operation 530, the electronic device (101) may calculate a preference for the processed image based on a user action. In one embodiment, the electronic device (101) may identify a preference associated with setting information for each of the plurality of images based on a user action for the images.

[0125] In one embodiment, the electronic device (101) can identify the preference of some images on which a user action has been performed among a plurality of images for the same object. In one embodiment, the electronic device (101) can identify a preference rank among the plurality of images for the same object based on the user action. In one embodiment, the electronic device (101) can identify the preference of some images on which a user action has been performed as being higher or lower than the preference of the remaining images on which a user action has been performed for the same object. In one embodiment, the electronic device (101) can identify the preference of an image on which a user action for deleting the image has been performed as being lower than the preference of other images for the same object (or lower than a reference preference). In one embodiment, the electronic device (101) may identify the preference of an image on which a user action to share, upload, change the storage location, and / or view the image has been performed as being higher than the preference of other images for the same object (or exceeding a reference preference). In one embodiment, the electronic device (101) may quantify the preference and / or identify a preference ranking of some images on which a user action has been performed based on clustering and / or regression analysis. In one embodiment, the clustering may be an algorithm for classifying a plurality of images into one or more preference groups. In one embodiment, one or more images included in a preference group may be evaluated as having the same and / or similar preference, but is not limited thereto. The electronic device (101) may identify the preference associated with the configuration information of each of the output images (320) based on text and / or audio input associated with the output image.For example, if there is a positive evaluation in the content of text and / or audio input related to the output image, the electronic device (101) may identify the preference related to the setting information of each of the output images (320) as being relatively high. For example, if there is a negative evaluation in the content of text and / or audio input related to the output image, the electronic device (101) may identify the preference related to the setting information of each of the output images (320) as being relatively low. For example, the electronic device (101) may identify the user action for viewing a plurality of images (321, 323, 325, 327, 329). For example, the electronic device (101) may identify a preference for the plurality of images (321, 323, 325, 327, 329) based on the user action for viewing the plurality of images (321, 323, 325, 327, 329). For example, the electronic device (101) may identify a preference that is proportional to the number of user inputs (or the number of viewings) for viewing the plurality of images (321, 323, 325, 327, 329).

[0126] In one embodiment, the electronic device (101) can identify the preference of some images on which a user action has been performed among a plurality of images for the same composition. In one embodiment, the electronic device (101) can identify the preference of some images on which a user action has been performed among a plurality of images for the same type. In one embodiment, the electronic device (101) can identify the preference of setting information for some images on which image processing has been performed according to an object, composition, and type.

[0127] In operation 540, the electronic device (101) may train a preference learning model (275) based on the derived preferences. In one embodiment, the electronic device (101) may train the preference learning model (275) based on the plurality of images for which the preferences are identified based on the user action. In one embodiment, the electronic device (101) may train the preference learning model (275) such that the preferences for images derived by the preference learning model (275) have the same and / or similar preferences as the preferences derived by the user action. In one embodiment, the preference learning model (275) may be an AI model that outputs preferences for input images. In one embodiment, the preference learning model (275) may be trained to output preferences for output images of the generative AI model (240) that are the same and / or similar to the preferences identified by the user action.

[0128] As described above, the electronic device (101) can repeat the learning for the preference learning model (275). In one embodiment, the electronic device (101) can re-perform the learning for the preference learning model (275) based on the accumulated number of images for which the user's preference is evaluated by the user's action. In one embodiment, the electronic device (101) can perform the learning for the preference learning model (275) based on the preference of the images according to the newly acquired user action.

[0129] FIG. 6A is a flowchart illustrating the operation of an electronic device according to one embodiment.

[0130] FIG. 6A may be described with reference to FIGS. 1 to 4D. Operation 603 of FIG. 6A may be included in operation 520 of FIG. 5. Operations 605, 630, and 640 of FIG. 6A may be included in operation 530 of FIG. 5.

[0131] Referring to FIG. 6A, in operation 601, the electronic device (101) may acquire similar images. In one embodiment, the similar images may be images with the same (or similar) composition. In one embodiment, the similar images may be images containing the same (or similar) objects. In one embodiment, the similar images may be images in succession.

[0132] In operation 603, the electronic device (101) may identify a user action for the similar images. For example, the user action may include deleting some of the similar images. For example, the user action may include saving some of the similar images. For example, saving the images may include changing the folder location.

[0133] In operation 605, the electronic device (101) can determine whether an image has been stored.

[0134] In operation 605, if it is determined that the image is stored, the electronic device (101) may perform operation 630. In operation 605, if it is determined that the image is not stored, the electronic device (101) may perform operation 640.

[0135] In operation 630, the electronic device (101) may identify a preferred image. In one embodiment, the electronic device (101) may identify an image determined to be stored among similar images as having a higher preference. In one embodiment, the electronic device (101) may identify an image determined to be stored among similar images as having a higher preference than the remaining images.

[0136] In operation 640, the electronic device (101) may identify an image as an unpreferred image. In one embodiment, the electronic device (101) may identify an image determined as not being stored among similar images as having a lower preference. In one embodiment, the electronic device (101) may identify an image determined as not being stored among similar images as having a lower preference than images determined as being stored. In one embodiment, the electronic device (101) may identify an image determined as not being stored among similar images as having a higher preference than images determined as being deleted.

[0137] FIG. 6b is a flowchart illustrating the operation of an electronic device according to one embodiment.

[0138] FIG. 6B may be described with reference to FIGS. 1 to 4D. Operation 611 of FIG. 6A may be included in operation 520 of FIG. 5. Operations 613, 630, and 640 of FIG. 6A may be included in operation 530 of FIG. 5.

[0139] Referring to FIG. 6B, in operation 611, the electronic device (101) may identify a user action for an image. In one embodiment, the user action for the image may include a user action for sharing (e.g., a user action for uploading to an application for a social network service (SNS)).

[0140] In operation 613, the electronic device (101) can determine whether an image has been uploaded. The electronic device (101) can determine, through a user action, whether an image has been uploaded to an application for SNS.

[0141] In operation 613, if it is determined that the image has been uploaded, the electronic device (101) may perform operation 630. In operation 613, if it is determined that the image has not been uploaded, the electronic device (101) may perform operation 640.

[0142] In operation 630, the electronic device (101) may identify a preferred image. In one embodiment, the electronic device (101) may identify an image determined to have been uploaded among the images as having a higher preference. In one embodiment, the electronic device (101) may identify an image determined to have been uploaded among the images as having a higher preference than the remaining images.

[0143] In operation 640, the electronic device (101) may identify an image as a non-preferred image. In one embodiment, the electronic device (101) may identify an image determined to not be uploaded among the images as having a lower preference. In one embodiment, the electronic device (101) may identify an image determined to not be uploaded among the images as having a lower preference than images determined to be stored.

[0144] FIG. 6c is a flowchart illustrating the operation of an electronic device according to one embodiment.

[0145] FIG. 6c can be described with reference to FIGS. 1 to 4d. Operation 620 of FIG. 6a can be included in operation 520 of FIG. 5. Operation 630 of FIG. 6a can be included in operation 530 of FIG. 5.

[0146] Referring to FIG. 6C, at operation 620, the electronic device (101) may identify a user action for image correction. In one embodiment, the user action related to image correction may include a user input for applying a filter to an image. Here, the filter may be a filter for applying a watercolor effect, brush strokes, blurring, distortion, or pixelation to the image.

[0147] In operation 630, the electronic device (101) may identify a preferred image. In one embodiment, the electronic device (101) may identify an image among the images that has been determined to have been corrected through a user action as having a higher preference. In one embodiment, the electronic device (101) may identify an image among the images that has been determined to have been corrected through a user action as having a higher preference than the remaining images.

[0148] Figure 7 is a flowchart illustrating the operation of an electronic device according to one embodiment.

[0149] Figure 7 can be explained with reference to Figures 1 to 4d.

[0150] Referring to FIG. 7, in operation 710, the electronic device (101) may perform image processing on an image. In one embodiment, the image may be a raw image acquired through a camera (280). In one embodiment, the image may be an image-processed image acquired from an external source (e.g., an electronic device (102, 103) or a server (108)) through a communication circuit (290). However, the present invention is not limited thereto. The image may be an image processed through a generative AI model (240) on a raw image acquired through the camera (280).

[0151] In one embodiment, the electronic device (101) may generate at least one output image (320) in which image processing is performed on input images (310) based on a prompt. In one embodiment, the electronic device (101) may generate at least one output image in which setting information of the input image is changed based on the prompt. In one embodiment, the setting information may include a plurality of setting items. In one embodiment, the setting information may include information about a focal length, a flash, an ISO sensitivity, an aperture, a shutter speed, a rotation direction, a color temperature, and / or a color space, but is not limited thereto. In one embodiment, the setting information may indicate whether hot pixel correction, demosaicing, color interpolation, noise reduction, lens shading correction, bad pixel processing, green imbalance correction, crosstalk compensation, or auto dark level compensation is performed. In one embodiment, the setting information may indicate whether edge enhancement, gamma correction, color saturation correction, or chromaticity correction is performed. In one embodiment, the setup information may indicate whether to perform auto white balance, auto focus, auto exposure, HDR, tone curve adjustment, resolution adjustment, and / or format conversion.

[0152] In one embodiment, the electronic device (101) may output output images (320) on which image processing has been performed based on different prompts (330) depending on the composition, type, and / or object of the input images (310). For example, output images (321, 323, 325) on which the same image processing has been performed may be generated for input images (311, 313, 315) on which the composition, type, and object have the same composition, type, and object. For example, output images (317, 319) on which the same image processing has been performed may be generated for input images (317, 319) on which the type and object have the same composition, type, and object. However, the present invention is not limited thereto.

[0153] In one embodiment, the electronic device (101) may generate prompts (330) including prompts related to composition, prompts related to type, and / or prompts related to objects. Each of the prompts related to composition, type, and object may have different targets that require image processing. For example, the prompts related to composition may indicate prompts related to gamma correction, color saturation correction, or chromaticity correction. For example, the prompts related to type may indicate prompts related to auto white balance, HDR, or edge enhancement. For example, the prompts related to objects may indicate prompts related to tone curve adjustment and resolution adjustment, but are not limited thereto.

[0154] In one embodiment, the electronic device (101) may generate a prompt (330) for each of the input images (310). For example, the prompt generation model (210) may generate a prompt (330) for image (311), a prompt (330) for image (313), a prompt (330) for image (315), a prompt (330) for image (317), and a prompt (330) for image (319). The prompt (330) for each of the input images (310) may include a prompt related to composition, a prompt related to type, and / or a prompt related to object. For example, for images (311, 313, and 315) of the same composition, the prompts related to composition may be similar. For example, for images (311, 313, 315, 317, 319) of the same type (e.g., pictures of animals), the prompts related to the type may be similar. For example, for images (311, 313, 315) of the same object (e.g., the same dog), the prompts related to the object may be similar.

[0155] In operation 720, the electronic device (101) can calculate a preference for the processed image based on the preference learning model (275).

[0156] In one embodiment, the electronic device (101) may output preferences (440) for output images (430) of the generative AI model (240) based on the preference learning model (275). In one embodiment, the electronic device (101) may output preferences (440) for each of the output images (430) of the generative AI model (240) based on the preference learning model (275). For example, the electronic device (101) may output preferences (440) for the output image (431), preferences (440) for the output image (433), preferences (440) for the output image (435), preferences (440) for the output image (437), and preferences (440) for the output image (439) based on the preference learning model (275). In one embodiment, the preference learning model (275) in operation 720 may be a pre-trained AI model or an AI model fine-tuned by user actions from a pre-trained AI model.

[0157] In operation 730, the electronic device (101) may train the prompt generation model (210) based on the calculated preferences. In one embodiment, the electronic device (101) may train the prompt generation model (210) based on the preferences (440). In one embodiment, the electronic device (101) may fine-tune the prompt generation model (210) based on the preferences (440). In one embodiment, the preferences (440) for the output images (430) of the preference learning model (275) may be understood as preferences for the prompts (450) of the prompt generation model (210) for guiding the generation of the output images.

[0158] In one embodiment, the electronic device (101) may fine-tune the prompt generation model (210) to generate another prompt to guide the generative AI model (240) in generating output images having setting information that yields high preferences based on the preferences (440) for the output images (430) of the preference learning model (275).

[0159] In one embodiment, the electronic device (101) can calculate a loss according to the preference (440). In one embodiment, the reinforcement learning model (270) can update the parameters of the prompt generation model (210) based on the loss. In one embodiment, the electronic device (101) can update the parameters of the prompt generation model (210) by backpropagating the loss. In one embodiment, the loss can be a negative preference (440), but is not limited thereto.

[0160] Figure 8 is a flowchart illustrating the operation of an electronic device according to one embodiment.

[0161] Figure 8 can be explained with reference to Figures 1 to 4d.

[0162] Referring to FIG. 8, in operation 810, the electronic device (101) may perform image processing on an image. In one embodiment, operation 810 may correspond to operation 710 of FIG. 7.

[0163] In operation 820, the electronic device (101) may calculate a preference for the processed image based on the preference learning model (275). In one embodiment, operation 820 may correspond to operation 720 of FIG. 7.

[0164] In operation 830, the electronic device (101) can train a generative AI model (240) based on the generated preferences.

[0165] In one embodiment, the electronic device (101) can train the generative AI model (240) based on the preference (440). In one embodiment, the electronic device (101) can fine-tune the generative AI model (240) based on the preference (440). In one embodiment, the electronic device (101) can fine-tune the generative AI model (240) based on the preference (440) for the output images (430) of the preference learning model (275) so that the generative AI model (240) generates output images having setting information that yields high preferences.

[0166] In one embodiment, the electronic device (101) can calculate a loss according to the preference (440). In one embodiment, the electronic device (101) can update the parameters of the generative AI model (240) based on the loss. In one embodiment, the electronic device (101) can update the parameters of the generative AI model (240) by backpropagating the loss. In one embodiment, the loss can be a negative preference (440), but is not limited thereto.

[0167] As described above, the electronic device (101) has been described as learning one of the prompt generation model (210) or the generative AI model (240) using the learned preference learning model (275), but this is merely an example. In one embodiment, the electronic device (101) may update the parameters of the prompt generation model (210) and / or the generative AI model (240) based on the loss according to the preference (440). In one embodiment, the electronic device (101) may update the parameters of the prompt generation model (210) and / or the generative AI model (240) by backpropagating the loss.

[0168] As described above, the electronic device (101) may include a processor (120). The electronic device (101) may include a memory (130) that stores instructions. The instructions, when executed by the processor (120), may cause the electronic device (101) to generate a plurality of images (320) through image processing of a plurality of raw images (310) by inputting a prompt (330) generated by the prompt generation model (210) into a generative AI model. The instructions, when executed by the processor (120), may cause the electronic device (101) to identify preferences associated with setting information of each of the plurality of images (321, 327, 329) based on a user action (401) with respect to the plurality of images (320). The instructions, when executed by the processor (120), may cause the electronic device (101) to fine tune the prompt generation model (210) based on the preferences, such that the generative AI model generates another prompt (450) to guide the generation of a plurality of other images (430) having setting information for which a high preference is derived.

[0169] The instructions, when executed by the processor (120), may cause the electronic device (101) to train a preference learning model for calculating a preference for each of the plurality of other images (430) based on the partial images (321, 327, 329) for which the preference has been identified based on the user action (401). The instructions, when executed by the processor (120), may cause the electronic device (101) to generate the plurality of other images (430) through image processing of the plurality of other raw images by inputting the other prompt (450) generated by the prompt generation model (210) into the generative AI model. The above instructions, when executed by the processor (120), may cause the electronic device (101) to calculate different preferences for each of the plurality of different images (430) through the preference learning model. The above instructions, when executed by the processor (120), may cause the electronic device (101) to fine-tune the prompt generation model (210) based on the calculated different preferences.

[0170] The plurality of images (320) may include the same object. The instructions, when executed by the processor (120), may cause the electronic device (101) to identify, based on the user action (401) for deleting the first images among the partial images (321, 327, 329), that the preference of each of the first images is lower than or equal to a reference preference.

[0171] The above instructions, when executed by the processor (120), may cause the electronic device (101) to identify, based on the user action (401) for changing the storage path of the second images among the some images (321, 327, 329), that the preference of each of the second images exceeds a reference preference.

[0172] The electronic device (101) may further include a display (260). The instructions, when executed by the processor (120), may cause the electronic device (101) to identify the remaining images excluding the some images (321, 327, 329) among the plurality of images (320). The instructions, when executed by the processor (120), may cause the electronic device (101) to display a user interface (UI) requesting an evaluation of a specified number of other some images (321, 327, 329) among the remaining images through the display (260). The above instructions, when executed by the processor (120), may cause the electronic device (101) to identify the preference associated with the setting information of each of the other some images (321, 327, 329) based on obtaining the evaluation for the other some images (321, 327, 329) through the UI.

[0173] The above instructions, when executed by the processor (120), may cause the electronic device (101) to identify the preference associated with the setting information of each of the partial images (321, 327, 329) based on the user action (401) for viewing the partial images (321, 327, 329). The preference based on the user action (401) for viewing the partial images (321, 327, 329) may be proportional to the number of times the partial images (321, 327, 329) have been viewed.

[0174] The electronic device (101) may further include a communication circuit (290). When the instructions are executed by the processor (120), the electronic device (101) may, through the communication circuit (290), identify a user action (401) to share the some images (321, 327, 329) with another electronic device (101), and identify that the preference of each of the some images (321, 327, 329) exceeds a reference preference.

[0175] The above instructions, when executed by the processor (120), may cause the electronic device (101) to identify the preference associated with the setting information of each of the some images (321, 327, 329) based on text associated with the some images (321, 327, 329) transmitted to the other electronic device (101).

[0176] The above instructions, when executed by the processor (120), may cause the electronic device (101) to identify, based on the user action (401) for applying an image effect to some of the images (321, 327, 329) among the plurality of images (320), that the preference of each of the some images (321, 327, 329) exceeds a reference preference.

[0177] The above instructions, when executed by the processor (120), may cause the electronic device (101) to fine-tune the generative AI model based on the preference such that the generative AI model generates the plurality of other images (430) having the setting information for which the high preference is produced.

[0178] The method performed by the electronic device (101) as described above may include an operation of generating a plurality of images (320) through image processing of a plurality of raw images by inputting a prompt (330) generated by a prompt generation model (210) into a generative AI model. The method may include an operation of identifying a preference associated with setting information of each of the plurality of images (321, 327, 329) based on a user action (401) with respect to some of the images (321, 327, 329) among the plurality of images (320). The method may include an operation of fine-tuning the prompt generation model (210) based on the preferences so that the generative AI model generates another prompt (450) to guide the generation of another plurality of images (430) having setting information for which a high preference is produced.

[0179] As described above, a non-transitory computer readable storage medium can store a program including instructions. The instructions, when executed by a processor (120) of an electronic device (101), can cause the electronic device (101) to generate a plurality of images (320) through image processing of a plurality of raw images (310) by inputting a prompt (330) generated by a prompt generation model (210) into a generative AI model. The instructions, when executed by the processor (120), may cause the electronic device (101) to identify preferences associated with setting information of each of the plurality of images (321, 327, 329) based on a user action (401) with respect to the plurality of images (320). The instructions, when executed by the processor (120), may cause the electronic device (101) to fine tune the prompt generation model (210) based on the preferences, such that the generative AI model generates another prompt (450) to guide the generation of a plurality of other images (430) having setting information for which a high preference is derived.

[0180] As described above, the electronic device (101) may include a processor (120). The electronic device (101) may include a memory (130) that stores instructions. The instructions, when executed by the processor (120), may cause the electronic device (101) to generate a plurality of images (320) through image processing of a plurality of raw images (310) by inputting a prompt (330) generated by the prompt generation model (210) into a generative AI model. The instructions, when executed by the processor (120), may cause the electronic device (101) to identify a preference associated with setting information of each of the plurality of images (321, 327, 329) based on a user action (401) with respect to the plurality of images (320). The instructions, when executed by the processor (120), may cause the electronic device (101) to fine tune the generative AI model based on the preference such that the generative AI model generates another plurality of images (430) having the setting information for which the high preference is produced.

[0181] The instructions, when executed by the processor (120), may cause the electronic device (101) to train a preference learning model for calculating a preference for each of the plurality of other images (430) based on the partial images (321, 327, 329) for which the preference has been identified based on the user action (401). The instructions, when executed by the processor (120), may cause the electronic device (101) to generate the plurality of other images (430) through image processing of the plurality of other raw images by inputting the other prompt (450) generated by the prompt generation model (210) into the generative AI model. The above instructions, when executed by the processor (120), may cause the electronic device (101) to calculate different preferences for each of the plurality of different images (430) through the preference learning model. The above instructions, when executed by the processor (120), may cause the electronic device (101) to fine-tune the generative AI model based on the calculated different preferences.

[0182] The plurality of images (320) may include the same object. The instructions, when executed by the processor (120), may cause the electronic device (101) to identify, based on the user action (401) for deleting the first images among the partial images (321, 327, 329), that the preference of each of the first images is lower than or equal to a reference preference.

[0183] The above instructions, when executed by the processor (120), may cause the electronic device (101) to identify, based on the user action (401) for changing the storage path of the second images among the some images (321, 327, 329), that the preference of each of the second images exceeds a reference preference.

[0184] The electronic device (101) may further include a display (260). The instructions, when executed by the processor (120), may cause the electronic device (101) to identify the remaining images excluding the some images (321, 327, 329) among the plurality of images (320). The instructions, when executed by the processor (120), may cause the electronic device (101) to display a user interface (UI) requesting an evaluation of a specified number of other some images (321, 327, 329) among the remaining images through the display (260). The above instructions, when executed by the processor (120), may cause the electronic device (101) to identify the preference associated with the setting information of each of the other some images (321, 327, 329) based on obtaining the evaluation for the other some images (321, 327, 329) through the UI.

[0185] The above instructions, when executed by the processor (120), may cause the electronic device (101) to identify the preference associated with the setting information of each of the partial images (321, 327, 329) based on the user action (401) for viewing the partial images (321, 327, 329). The preference based on the user action (401) for viewing the partial images (321, 327, 329) may be proportional to the number of times the partial images (321, 327, 329) have been viewed.

[0186] The electronic device (101) may further include a communication circuit (290). The instructions, when executed by the processor (120), may cause the electronic device (101) to identify, through the communication circuit (290), a user action (401) for sharing the partial images (321, 327, 329) with another electronic device (101), and to identify that the preference of each of the partial images (321, 327, 329) exceeds a reference preference.

[0187] The above instructions, when executed by the processor (120), may cause the electronic device (101) to identify the preference associated with the setting information of each of the some images (321, 327, 329) based on text associated with the some images (321, 327, 329) transmitted to the other electronic device (101).

[0188] The above instructions, when executed by the processor (120), may cause the electronic device (101) to identify, based on the user action (401) for applying an image effect to some of the images (321, 327, 329) among the plurality of images (320), that the preference of each of the some images (321, 327, 329) exceeds a reference preference.

[0189] The method performed by the electronic device (101) as described above may include an operation of generating a plurality of images (320) through image processing of a plurality of raw images by inputting a prompt (330) generated by a prompt generation model (210) into a generative AI model. The method may include an operation of identifying a preference associated with setting information of each of the plurality of images (321, 327, 329) based on a user action (401) with respect to some of the images (321, 327, 329) among the plurality of images (320). The method may include an operation of fine-tuning the generative AI model based on the preferences such that the generative AI model generates other images (430) having the setting information from which the high preference is derived.

[0190] As described above, a non-transitory computer readable storage medium can store a program including instructions. The instructions, when executed by a processor (120) of an electronic device (101), can cause the electronic device (101) to generate a plurality of images (320) through image processing of a plurality of raw images (310) by inputting a prompt (330) generated by a prompt generation model (210) into a generative AI model. The instructions, when executed by the processor (120), may cause the electronic device (101) to identify a preference associated with setting information of each of the plurality of images (321, 327, 329) based on a user action (401) with respect to the plurality of images (320). The instructions, when executed by the processor (120), may cause the electronic device (101) to fine tune the generative AI model based on the preference such that the generative AI model generates another plurality of images (430) having the setting information for which the high preference is produced.

[0191] Electronic devices according to the various embodiments disclosed in this document may take various forms. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. Electronic devices according to the embodiments of this document are not limited to the aforementioned devices.

[0192] The various embodiments of this document and the terminology used therein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly indicates otherwise. In this document, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" can include any one of the items listed together in the corresponding phrase among those phrases, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used merely to distinguish one component from another, and do not limit the components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as "coupled" or "connected" to another component (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0193] The term "module" used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or part of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0194] Various embodiments of the present document may be implemented as software (e.g., a program (140)) including one or more instructions stored in a storage medium (e.g., an internal memory (136) or an external memory (138)) readable by a machine (e.g., an electronic device (101)). For example, a processor (e.g., a processor (120)) of the machine (e.g., an electronic device (101)) may call at least one instruction among the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' simply means that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently or temporarily on the storage medium.

[0195] According to one embodiment, the method according to various embodiments disclosed in this document may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., a compact disc read-only memory (CD-ROM)), or may be distributed online (e.g., by download or upload) through an application store (e.g., Play Store™) or directly between two user devices (e.g., smart phones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0196] According to various embodiments, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separated and placed in other components. According to various embodiments, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to various embodiments, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

Claims

1. In an electronic device (101), processor (120), and It includes a memory (130) that stores instructions, and when the instructions are executed by the processor (120), the electronic device (101) By inputting the prompt (330) generated by the prompt generation model (210) into the generative AI (artificial intelligence) model, a plurality of images (320) are generated through image processing of a plurality of raw images (310). Based on a user action (401) for some images (321, 327, 329) among the above plurality of images (320), preferences related to setting information of each of the above images (321, 327, 329) are identified, Causing the generative AI model to fine tune the prompt generation model (210) based on the preference to generate another prompt (450) to guide the generation of a plurality of other images (430) having setting information that yields high preference. Electronic devices.

2. In claim 1, The above instructions, when executed by the processor (120), cause the electronic device (101) to: Based on the user action (401) above, a preference learning model is trained to derive preferences for each of the plurality of other images (430) based on the images (321, 327, 329) for which preferences have been identified. By inputting the other prompt (450) generated by the above prompt generation model (210) into the generative AI model, a plurality of other images (430) are generated through image processing of a plurality of other raw images, Through the above preference learning model, different preferences for each of the above multiple different images (430) are obtained, Based on the other preferences acquired above, causing the prompt generation model (210) to be fine-tuned, Electronic devices.

3. In claim 1 or claim 2, The above multiple images (320) contain the same object, The above instructions, when executed by the processor (120), cause the electronic device (101) to: Based on the user action (401) to delete the first images among the above some images (321, 327, 329), causing each of the first images to be identified as having a preference lower than or equal to a reference preference, Electronic devices.

4. In claim 3, The above instructions, when executed by the processor (120), cause the electronic device (101) to: Based on the user action (401) for changing the storage path of the second images among the above some images (321, 327, 329), causing each of the second images to be identified as having a preference exceeding a reference preference, Electronic devices.

5. In any one of claims 1 to 4, Further including a display (260), The above instructions, when executed by the processor (120), cause the electronic device (101) to: Identify the remaining images excluding some of the images (321, 327, 329) among the above multiple images (320), A UI (user interface) requesting evaluation of a specified number of other images (321, 327, 329) among the remaining images is displayed through the display (260), Based on obtaining the evaluation for said other some images (321, 327, 329) through said UI, causing said preference related to said setting information of each of said other some images (321, 327, 329) to be identified. Electronic devices.

6. In any one of claims 1 to 5, The above instructions, when executed by the processor (120), cause the electronic device (101) to: Based on the user action (401) for viewing the above-mentioned some images (321, 327, 329), the preference related to the setting information of each of the above-mentioned some images (321, 327, 329) is caused to be identified, and the preference based on the user action (401) for viewing the above-mentioned some images (321, 327, 329) is proportional to the number of times the above-mentioned some images (321, 327, 329) are viewed. Electronic devices.

7. In claim 1, Further comprising a communication circuit (290), The above instructions, when executed by the processor (120), cause the electronic device (101) to: Through the above communication circuit (290), based on the identification of a user action (401) for sharing the some images (321, 327, 329) with another electronic device (101), causing each of the some images (321, 327, 329) to be identified as exceeding a reference preference, Electronic devices.

8. In claim 7, The above instructions, when executed by the processor (120), cause the electronic device (101) to: Based on the text related to said some images (321, 327, 329) transmitted to said other electronic device (101), causing said preference related to said setting information of each of said some images (321, 327, 329) to be identified. Electronic devices.

9. In any one of claims 1 to 8, The above instructions, when executed by the processor (120), cause the electronic device (101) to: Based on the user action (401) for applying an image effect to some images (321, 327, 329) among the plurality of images (320), causing each of the some images (321, 327, 329) to be identified as having a preference exceeding a reference preference, Electronic devices.

10. In any one of claims 1 to 9, The above instructions, when executed by the processor (120), cause the electronic device (101) to: Causing the generative AI model to fine-tune the generative AI model based on the preference so as to generate a plurality of other images (430) having the setting information for which the high preference is obtained. Electronic devices.

11. In an electronic device (101), processor (120), and It includes a memory (130) that stores instructions, and when the instructions are executed by the processor (120), the electronic device (101) By inputting the prompt (330) generated by the prompt generation model (210) into the generative AI model, a plurality of images (320) are generated through image processing of a plurality of raw images. Based on a user action (401) for some images (321, 327, 329) among the above plurality of images (320), preferences related to setting information of each of the above images (321, 327, 329) are identified, Causing the generative AI model to fine tune the generative AI model based on the preference so as to generate a plurality of other images (430) having the setting information for which the high preference is obtained. Electronic devices.

12. In claim 11, The above instructions, when executed by the processor (120), cause the electronic device (101) to: Based on the user action (401) above, a preference learning model is trained to obtain preferences for each of the plurality of other images (430) based on the images (321, 327, 329) for which preferences are identified. By inputting the other prompt (450) generated by the above prompt generation model (210) into the generative AI model, a plurality of other images (430) are generated through image processing of a plurality of other raw images, Through the above preference learning model, different preferences for each of the above multiple different images (430) are obtained, Based on the other preferences acquired above, causing the generative AI model to be fine-tuned, Electronic devices.

13. In claim 11 or claim 12, The above multiple images (320) contain the same object, The above instructions, when executed by the processor (120), cause the electronic device (101) to: Based on the user action (401) to delete the first images among the above some images (321, 327, 329), causing each of the first images to be identified as having a preference lower than or equal to a reference preference, Electronic devices.

14. In claim 13, The above instructions, when executed by the processor (120), cause the electronic device (101) to: Based on the user action (401) for changing the storage path of the second images among the above some images (321, 327, 329), causing each of the second images to be identified as having a preference exceeding a reference preference, Electronic devices.

15. In a method performed by an electronic device (101), An operation of generating multiple images (320) by inputting a prompt (330) generated by a prompt generation model (210) into a generative AI model, through image processing of multiple raw images. An operation of identifying preferences related to setting information of each of the images (321, 327, 329) based on a user action (401) for some of the images (321, 327, 329) among the plurality of images (320), and An operation of fine tuning the prompt generation model (210) based on the preference to generate another prompt (450) to guide the generation of a plurality of other images (430) having setting information for which the generative AI model obtains high preference. method.

Images