Method for display preview image by using multi-view camera, electronic device supporting same, and storage medium

Abstract

An electronic device according to one embodiment may comprise a first image sensor, a second image sensor, a touch screen display, at least one processor and a memory for storing instructions. When executed individually or collectively by the at least one processor, the instructions can instruct the electronic device to: acquire a first preview image through the first image sensor; acquire a second preview image through the second image sensor; display, by means of the touch screen display, the first preview image on a first portion of the touchscreen display and display the second preview image on a second portion of the touchscreen display; receive a user input for generating an AI object for the first preview image; on the basis of the user input, generate the AI object by using a machine learning model, the machine learning model including a generative AI model trained to output at least one AI object on the basis of text, voice or drawing input; on the basis of the characteristics of the generated AI object, use the machine learning model so as to modify the first preview image and the second preview image; and display the modified first preview image on the first portion and display the modified second preview image on the second portion.

Description

A method for displaying a preview image using a multiview camera, an electronic device supporting the same, and a storage medium

[0001] The present disclosure relates to a method for displaying a preview image using a multiview camera, an electronic device supporting the same, and a storage medium.

[0002] Driven by the remarkable advancements in information and communication technology and semiconductor technology, the distribution and use of various electronic devices are increasing rapidly. Electronic devices are being developed to enable users to carry them around and communicate. The term "electronic device" may refer to a device that performs specific functions according to an installed program, such as mobile communication terminals, tablet PCs, video / audio devices, desktop / laptop computers, or in-vehicle navigation systems.

[0003] The electronic device can acquire an image through a camera and display the acquired image (e.g., a preview image) through a display. The electronic device provides an image (e.g., a captured image) or video upon the user's request.

[0004] The information described above may be provided as related art for the purpose of aiding understanding of the present disclosure. No claim or determination is made as to whether any of the foregoing may be applied as prior art related to the present disclosure.

[0005] An electronic device according to one embodiment may include a first image sensor, a second image sensor, a touchscreen display, at least one processor, and a memory for storing instructions. When the instructions are executed individually or collectively by the at least one processor, the electronic device may cause the first preview image to be acquired through the first image sensor. When the instructions are executed individually or collectively by the at least one processor, the electronic device may cause the second preview image to be acquired through the second image sensor. When the instructions are executed individually or collectively by the at least one processor, the electronic device may cause the first preview image to be displayed on a first part of the touchscreen display and the second preview image to be displayed on a second part of the touchscreen display through the touchscreen display. When the above instructions are executed individually or collectively by the at least one processor, they may cause the electronic device to receive user input for generating an AI object for the first preview image. When the above instructions are executed individually or collectively by the at least one processor, they may cause the electronic device to generate the AI ​​object using a machine learning model based on the user input. The machine learning model may include a generative AI model trained to output at least one AI object based on text, voice, or drawing input.When the above instructions are executed individually or collectively by the at least one processor, they may cause the electronic device to modify the first preview image and the second preview image using the machine learning model based on the characteristics of the generated AI object. When the above instructions are executed individually or collectively by the at least one processor, they may cause the electronic device to display the modified first preview image in the first part and display the modified second preview image in the second part.

[0006] A method according to one embodiment may include an operation of acquiring a first preview image through a first image sensor of an electronic device. The method may include an operation of acquiring a second preview image through a second image sensor of the electronic device. The method may include an operation of displaying the first preview image on a first part of a touchscreen display and displaying the second preview image on a second part of a touchscreen display through a touchscreen display of the electronic device. The method may include an operation of receiving user input for generating an AI object with respect to the first preview image. The method may include an operation of generating the AI ​​object using a machine learning model based on the user input. The machine learning model may include a generative AI model trained to output at least one AI object based on text, voice, or drawing input. The method may include an operation of modifying the first preview image and the second preview image using the machine learning model based on the characteristics of the generated AI object. The above method may include the operation of displaying the modified first preview image in the first part and displaying the modified second preview image in the second part.

[0007] According to one embodiment, a computer-readable medium recording computer-executable instructions may be provided. When executed by at least one processor of an electronic device, the computer-executable instructions may cause the electronic device to acquire a first preview image through a first image sensor of the electronic device. When executed by at least one processor of an electronic device, the computer-executable instructions may cause the electronic device to acquire a second preview image through a second image sensor of the electronic device. When executed by at least one processor of an electronic device, the computer-executable instructions may cause the electronic device to display the first preview image on a first part of the touchscreen display and display the second preview image on a second part of the touchscreen display through the touchscreen display of the electronic device. When executed by at least one processor of an electronic device, the computer-executable instructions may cause the electronic device to receive user input for creating an AI object with respect to the first preview image. When the above computer-executable instructions are executed by at least one processor of the electronic device, the electronic device may cause the electronic device to generate the AI ​​object using a machine learning model based on the user input. The machine learning model may include a generative AI model trained to output at least one AI object based on text, voice, or drawing input. When the above computer-executable instructions are executed by at least one processor of the electronic device, the electronic device may cause the first preview image and the second preview image to modify the machine learning model based on the characteristics of the generated AI object.When the above computer-executable instructions are executed by at least one processor of an electronic device, the electronic device may cause the modified first preview image to be displayed in the first part and the modified second preview image to be displayed in the second part.

[0008] An electronic device according to one embodiment may include a first image sensor, a second image sensor, a touchscreen display, at least one processor, and a memory for storing instructions. When the instructions are executed individually or collectively by the at least one processor, the electronic device may cause the first preview image to be acquired through the first image sensor. When the instructions are executed individually or collectively by the at least one processor, the electronic device may cause the second preview image to be acquired through the second image sensor. When the instructions are executed individually or collectively by the at least one processor, the electronic device may cause the first preview image to be displayed on a first part of the touchscreen display and the second preview image to be displayed on a second part of the touchscreen display through the touchscreen display. When the above instructions are executed individually or collectively by the at least one processor, the electronic device may be caused to obtain a modified first preview image containing the first object based on information output from the generative AI model by inputting the first preview image to a generative AI model trained to display a graphic object on the input preview image based on receiving user input for displaying the first object on the first preview image. When the above instructions are executed individually or collectively by the at least one processor, the electronic device may be caused to obtain a modified second preview image based on information output from the generative AI model by inputting information associated with the characteristics of the first object and the second preview image to the generative AI model.When the above instructions are executed individually or collectively by the at least one processor, the electronic device may cause the modified first preview image to be displayed in the first part and the modified second preview image to be displayed in the second part through the touchscreen display.

[0009] A method according to one embodiment may include an operation of acquiring a first preview image through a first image sensor of an electronic device. The method may include an operation of acquiring a second preview image through a second image sensor of the electronic device. The method may include an operation of displaying the first preview image on a first part of the touchscreen display and displaying the second preview image on a second part of the touchscreen display through a touchscreen display of the electronic device. The method may include an operation of acquiring a modified first preview image containing the first object based on information output from a generative AI model by inputting the first preview image into a generative AI model trained to display a graphic object on the input preview image based on receiving user input for displaying a first object on the first preview image. The method may include an operation of acquiring a modified second preview image based on information output from a generative AI model by inputting information associated with the characteristics of the first object and the second preview image into the generative AI. The above method may include the operation of displaying the modified first preview image in the first part and displaying the modified second preview image in the second part through the touchscreen display.

[0010] According to one embodiment, a computer-readable medium recording computer-executable instructions may be provided. When executed by at least one processor of an electronic device, the computer-executable instructions may cause the electronic device to acquire a first preview image through a first image sensor of the electronic device. When executed by at least one processor of an electronic device, the computer-executable instructions may cause the electronic device to acquire a second preview image through a second image sensor of the electronic device. When executed by at least one processor of an electronic device, the computer-executable instructions may cause the electronic device to display the first preview image on a first part of the touchscreen display and display the second preview image on a second part of the touchscreen display through the touchscreen display of the electronic device. When the above computer-executable instructions are executed by at least one processor of an electronic device, the electronic device may be caused to obtain a modified first preview image containing the first object based on information output from a generative AI model by inputting the first preview image to a generative AI model trained to display a graphic object on the input preview image based on receiving user input for displaying a first object on the first preview image. When the above computer-executable instructions are executed by at least one processor of an electronic device, the electronic device may be caused to obtain a modified second preview image based on information output from a generative AI model by inputting information associated with the characteristics of the first object and the second preview image to the generative AI.When the above computer-executable instructions are executed by at least one processor of the electronic device, the electronic device may cause the modified first preview image to be displayed in the first part and the modified second preview image to be displayed in the second part through the touchscreen display.

[0011] An electronic device according to one embodiment may include a first image sensor, a second image sensor, a touchscreen display, at least one processor, and a memory for storing instructions. When the instructions are executed individually or collectively by the at least one processor, the electronic device may cause the first preview image to be acquired through the first image sensor. When the instructions are executed individually or collectively by the at least one processor, the electronic device may cause the second preview image to be acquired through the second image sensor. When the instructions are executed individually or collectively by the at least one processor, the electronic device may cause the first preview image to be displayed on a first part of the touchscreen display and the second preview image to be displayed on a second part of the touchscreen display through the touchscreen display. The above instructions, when executed individually or collectively by the at least one processor, may cause the electronic device to receive user input for creating at least one graphic object on the first preview image through the touchscreen display. The above instructions, when executed individually or collectively by the at least one processor, may cause the electronic device to create a graphic object on the first preview image based on at least a portion of the user input. The above instructions, when executed individually or collectively by the at least one processor, may cause the electronic device to display a modified first preview image containing the graphic object on a first portion of the touchscreen display through the touchscreen display, based on at least a portion of the characteristics of the graphic object.When the above instructions are executed individually or collectively by the at least one processor, they may cause the electronic device to modify the second preview image based on at least some of the characteristics of the graphic object. When the above instructions are executed individually or collectively by the at least one processor, they may cause the electronic device to display the modified second preview image on the second part through the touchscreen display.

[0012] A method according to one embodiment may include an operation of acquiring a first preview image through a first image sensor of an electronic device. The method may include an operation of acquiring a second preview image through a second image sensor of the electronic device. The method may include an operation of displaying the first preview image on a first part of the touchscreen display and displaying the second preview image on a second part of the touchscreen display through the touchscreen display of the electronic device. The method may include an operation of receiving user input for creating at least one graphic object for the first preview image through the touchscreen display. The method may include an operation of creating a graphic object on the first preview image based on at least a part of the user input. The method may include an operation of displaying a modified first preview image containing the graphic object on the first part of the touchscreen display through the touchscreen display based on at least a part of the characteristics of the graphic object. The method may include an operation of modifying the second preview image based on at least a part of the characteristics of the graphic object. The above method may include the operation of displaying the modified second preview image in the second part through the touchscreen display.

[0013] According to one embodiment, a computer-readable medium recording computer-executable instructions may be provided. When executed by at least one processor of an electronic device, the computer-executable instructions may cause the electronic device to acquire a first preview image through a first image sensor of the electronic device. When executed by at least one processor of an electronic device, the computer-executable instructions may cause the electronic device to acquire a second preview image through a second image sensor of the electronic device. When executed by at least one processor of an electronic device, the computer-executable instructions may cause the electronic device to display the first preview image on a first part of the touchscreen display and display the second preview image on a second part of the touchscreen display through the touchscreen display of the electronic device. When executed by at least one processor of an electronic device, the computer-executable instructions may cause the electronic device to receive user input for creating at least one graphic object for the first preview image through the touchscreen display. When the above computer-executable instructions are executed by at least one processor of the electronic device, the electronic device may cause the electronic device to generate a graphic object on the first preview image based on at least a portion of the user input. When the above computer-executable instructions are executed by at least one processor of the electronic device, the electronic device may cause the electronic device to display a modified first preview image containing the graphic object on a first portion of the touchscreen display through the touchscreen display, based on at least a portion of the characteristics of the graphic object.When the above computer-executable instructions are executed by at least one processor of the electronic device, the electronic device may cause the second preview image to be modified based on at least some of the characteristics of the graphic object. When the above computer-executable instructions are executed by at least one processor of the electronic device, the electronic device may cause the modified second preview image to be displayed on the second part through the touchscreen display.

[0014] In relation to the description of the drawings, the same or similar reference numerals may be used for identical or similar components.

[0015] FIG. 1 is a block diagram of an electronic device in a network environment according to one embodiment.

[0016] FIG. 2 is a block diagram of an electronic device according to one embodiment.

[0017] FIG. 3a is a drawing for explaining the location where a camera of an electronic device is placed, according to one embodiment.

[0018] FIGS. 3B and FIGS. 3C are drawings for illustrating a method of providing a preview image of an electronic device according to one embodiment.

[0019] FIG. 4 is a drawing for explaining a method of providing a modified preview image of an electronic device according to one embodiment.

[0020] FIG. 5 is a flowchart illustrating a method for providing a modified preview image of an electronic device according to one embodiment.

[0021] FIGS. 6a, FIGS. 6b, FIGS. 6c, and FIGS. 6d are drawings for illustrating a method of providing a modified preview image of an electronic device according to one embodiment.

[0022] FIGS. 7a and 7b are drawings for illustrating a method of providing modified image frames of an electronic device according to one embodiment.

[0023] FIG. 8 is a flowchart illustrating a method for providing a modified preview image of an electronic device according to one embodiment.

[0024] FIG. 9 is a drawing for illustrating a method of providing a modified preview image of an electronic device according to one embodiment.

[0025] FIG. 10 is a flowchart illustrating a method for providing a modified preview image of an electronic device according to one embodiment.

[0026] FIG. 11 is a flowchart illustrating a method for providing a modified preview image of an electronic device according to one embodiment.

[0027] FIG. 12 is a flowchart illustrating a method for providing a modified preview image of a foldable electronic device according to one embodiment.

[0028] FIG. 13a is a drawing for illustrating a foldable electronic device according to one embodiment.

[0029] FIG. 13b is a drawing illustrating a method for a foldable electronic device to provide a preview image according to one embodiment.

[0030] FIGS. 13c, FIGS. 13d, and FIGS. 13e are drawings for illustrating a method in which a foldable electronic device according to one embodiment provides a modified preview image.

[0031] FIG. 14 is a drawing illustrating a method for an electronic device to provide a preview image according to one embodiment.

[0032] FIG. 15 is a diagram illustrating a generative artificial intelligence model according to one embodiment.

[0033] Hereinafter, embodiments of the present disclosure are described in detail with reference to the drawings so that those skilled in the art can easily practice them. However, the present disclosure may be embodied in various different forms and is not limited to the embodiments described herein. In relation to the description of the drawings, the same or similar reference numerals may be used for identical or similar components. Furthermore, in the drawings and related descriptions, descriptions of well-known functions and configurations may be omitted for clarity and brevity.

[0034] FIG. 1 is a block diagram of an electronic device (101) in a network environment (100) according to one embodiment.

[0035] Referring to FIG. 1, in a network environment (100), an electronic device (101) may communicate with an electronic device (102) through a first network (198) (e.g., a short-range wireless communication network) or with at least one of an electronic device (104) or a server (108) through a second network (199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (101) may communicate with the electronic device (104) through a server (108). According to one embodiment, the electronic device (101) may include a processor (120), memory (130), input module (150), sound output module (155), display module (160), audio module (170), sensor module (176), interface (177), connection terminal (178), haptic module (179), camera module (180), power management module (188), battery (189), communication module (190), subscriber identification module (196), or antenna module (197). In some embodiments, at least one of these components (e.g., connection terminal (178)) may be omitted from the electronic device (101), or one or more other components may be added. In some embodiments, some of these components (e.g., sensor module (176), camera module (180), or antenna module (197)) may be integrated into a single component (e.g., display module (160)).

[0036] The processor (120) can control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) by executing software (e.g., a program (140)), and can perform various data processing or operations. According to one embodiment, as at least part of the data processing or operations, the processor (120) can store commands or data received from other components (e.g., a sensor module (176) or a communication module (190)) in volatile memory (132), process the commands or data stored in volatile memory (132), and store the resulting data in non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or an auxiliary processor (123) that can operate independently or together with it (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor). For example, if the electronic device (101) includes a main processor (121) and an auxiliary processor (123), the auxiliary processor (123) may be configured to use lower power than the main processor (121) or to be specialized for a designated function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as part thereof.

[0037] The auxiliary processor (123) may control at least some of the functions or states associated with at least one component of the electronic device (101) (e.g., display module (160), sensor module (176), or communication module (190)) on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. According to one embodiment, the auxiliary processor (123) (e.g., image signal processor or communication processor) may be implemented as part of another functionally related component (e.g., camera module (180) or communication module (190)). According to one embodiment, the auxiliary processor (123) (e.g., neural network processing unit) may include a hardware structure specialized for processing an artificial intelligence model. The artificial intelligence model may be generated through machine learning. Such learning may be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or through a separate server (e.g., server (108)). The learning algorithm may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model may include a plurality of artificial neural network layers.An artificial neural network may be a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a generative adversarial network (GAN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to the hardware structure, the artificial intelligence model may include a software structure, either additionally or substantially.

[0038] The memory (130) can store various data used by at least one component of the electronic device (101) (e.g., processor (120) or sensor module (176)). The data may include, for example, input data or output data for software (e.g., program (140)) and related commands. The memory (130) may include volatile memory (132) or non-volatile memory (134).

[0039] The program (140) may be stored as software in memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0040] The input module (150) can receive commands or data to be used for a component of the electronic device (101) (e.g., processor (120)) from outside the electronic device (101) (e.g., user). The input module (150) may include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0041] The sound output module (155) can output a sound signal to the outside of the electronic device (101). The sound output module (155) may include, for example, a speaker or a receiver. The speaker may be used for general purposes, such as multimedia playback or recording playback. The receiver may be used to receive incoming calls. According to one embodiment, the receiver may be implemented separately from the speaker or as part thereof.

[0042] The display module (160) can visually provide information to an external (e.g., user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling said device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of the force generated by said touch.

[0043] The audio module (170) can convert sound into an electrical signal or, conversely, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150) or output sound through the sound output module (155) or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphones) connected directly or wirelessly to the electronic device (101).

[0044] The sensor module (176) can detect the operating state of the electronic device (101) (e.g., power or temperature) or the external environmental state (e.g., user state) and generate an electrical signal or data value corresponding to the detected state. According to one embodiment, the sensor module (176) may include, for example, a gesture sensor, a gyroscope sensor, a barometric pressure sensor, a magnetic sensor, an accelerometer sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biosensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0045] The interface (177) may support one or more specified protocols that can be used for the electronic device (101) to be connected directly or wirelessly to an external electronic device (e.g., electronic device (102)). According to one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0046] The connection terminal (178) may include a connector through which the electronic device (101) can be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0047] The haptic module (179) can convert an electrical signal into a mechanical stimulus (e.g., vibration or movement) or an electrical stimulus that can be perceived by the user through tactile or kinesthetic senses. According to one embodiment, the haptic module (179) may include, for example, a motor, a piezoelectric element, or an electric stimulation device.

[0048] The camera module (180) can capture still images and video. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0049] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented, for example, as at least part of a power management integrated circuit (PMIC).

[0050] The battery (189) can supply power to at least one component of the electronic device (101). According to one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0051] The communication module (190) can support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between an electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may include one or more communication processors that operate independently of the processor (120) (e.g., application processor) and support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., cellular communication module, short-range wireless communication module, or GNSS (global navigation satellite system) communication module) or a wired communication module (194) (e.g., LAN (local area network) communication module, or power line communication module). The corresponding communication module among these communication modules can communicate with an external electronic device (104) through a first network (198) (e.g., a short-range communication network such as Bluetooth, WiFi (wireless fidelity) direct, or IrDA (infrared data association)) or a second network (199) (e.g., a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules may be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can identify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) using subscriber information (e.g., International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module (196).

[0052] The wireless communication module (192) can support 5G networks and next-generation communication technologies following 4G networks, for example, new radio access technology. NR access technology can support high-speed transmission of high-capacity data (enhanced mobile broadband (eMBB)), minimization of terminal power and connection of multiple terminals (massive machine type communications (mMTC)), or high reliability and low latency (ultra-reliable and low-latency communications (URLLC)). The wireless communication module (192) can support a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate, for example. The wireless communication module (192) can support various technologies for securing performance in the high-frequency band, such as beamforming, massive MIMO (multiple-input and multiple-output), full-dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large-scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), external electronic device (e.g., electronic device (104)), or network system (e.g., second network (199)). According to one embodiment, the wireless communication module (192) may support a Peak data rate (e.g., 20 Gbps or more) for eMBB realization, loss coverage (e.g., 164 dB or less) for mMTC realization, or U-plane latency (e.g., downlink (DL) and uplink (UL) each 0.5 ms or less, or round trip 1 ms or less) for URLLC realization.

[0053] An antenna module (197) can transmit a signal or power to or from an external source (e.g., an external electronic device). According to one embodiment, the antenna module (197) may include an antenna comprising a radiator made of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). According to one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as a first network (198) or a second network (199), may be selected from the plurality of antennas, for example, by a communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device through the selected at least one antenna. According to some embodiments, in addition to the radiator, other components (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as part of the antenna module (197).

[0054] According to various embodiments, the antenna module (197) may form a mmWave antenna module. According to one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent to a first surface (e.g., bottom surface) of the printed circuit board and capable of supporting a specified high frequency band (e.g., mmWave band), and a plurality of antennas (e.g., array antennas) disposed on or adjacent to a second surface (e.g., top surface or side surface) of the printed circuit board and capable of transmitting or receiving a signal of the specified high frequency band.

[0055] At least some of the above components can be connected to each other via a communication method between peripheral devices (e.g., bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)) and exchange signals (e.g., commands or data) with each other.

[0056] According to one embodiment, commands or data may be transmitted or received between an electronic device (101) and an external electronic device (104) through a server (108) connected to a second network (199). Each of the external electronic devices (102, or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations performed on the electronic device (101) may be performed on one or more of the external electronic devices (102, 104, or 108). For example, if the electronic device (101) needs to perform a function or service automatically or in response to a request from a user or another device, the electronic device (101) may request one or more external electronic devices to perform at least part of the function or service instead of performing the function or service itself or additionally. One or more external electronic devices that receive the above request may execute at least part of the requested function or service, or additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may provide the result as is or additionally processed as at least part of the response to the request. For this purpose, for example, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used. The electronic device (101) may provide ultra-low latency services using, for example, distributed computing or mobile edge computing. In one embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server using machine learning and / or neural networks. According to one embodiment, the external electronic device (104) or the server (108) may be included within a second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0057] FIG. 2 is a block diagram of an electronic device according to one embodiment.

[0058] In one embodiment, the electronic device (201) may be the electronic device (101) of FIG. 1.

[0059] Referring to FIG. 2, in one embodiment, the electronic device (201) may include a first image sensor (211), a second image sensor (213), a display (220), a memory (230), and / or a processor (240).

[0060] In one embodiment, the first image sensor (211) and / or the second image sensor (213) may be included in the camera module (180) of FIG. 1. The first image sensor (211) and / or the second image sensor (213) may acquire an image of a subject outside the electronic device (101).

[0061] In one embodiment, the display (220) may be included in the display module (160) of FIG. 1. In one embodiment, the display (220) may display a first preview image obtained by the first image sensor (211) and a second preview image obtained by the second image sensor (213). The preview images displayed through the display (220) will be described later.

[0062] In one embodiment, the memory (230) may be included in the memory (130) of FIG. 1.

[0063] In one embodiment, the memory (230) may store metadata of the preview image and information associated with an artificial intelligence (AI) object included in the preview image. In one embodiment, the memory (230) may store a generative artificial intelligence (AI) model. The generative AI model may include an artificial intelligence model trained to output content similar to the input content based on receiving content such as text, audio, and / or images. The generative AI model may learn, for example, patterns of content included in training data. The type of content output by the generative AI model may include at least one of an image, audio, or text associated with a description of the output content. In one embodiment, the operation of acquiring content using the generative AI model may be referred to as the “operation of generating content.” In one embodiment, the electronic device (201) may generate an image using the generative AI model. Here, the generative AI model can generate text, images, and other media in response to input prompts, and can generate and output new content rather than simply analyzing existing data. The generative AI model can be trained to recognize objects included in an image by extracting key features from a first preview image acquired by a first image sensor (211) and / or a second preview image acquired by a second image sensor (213), and to generate an image containing AI objects.

[0064] According to one embodiment, the AI ​​model (or generative AI model) may be an on-device AI model operating on an electronic device (201) or a server AI model operating on an external server (e.g., server (108) of FIG. 1). In one embodiment, the AI ​​model (or generative AI model) may be stored in an external electronic device. For example, the on-device AI model of the electronic device (201) may be used to perform at least some operations during the process of performing a conversation between the electronic device (201) and a user. According to one embodiment, the AI ​​model of an external electronic device (or external server) may be used to perform at least some operations during the process of performing a conversation between the electronic device (201) and a user. According to one embodiment, the on-device AI model of the electronic device (201) and the AI ​​model of the external electronic device may be used individually or collectively to perform at least some operations during the process of performing a conversation between the electronic device (201) and a user. For example, the operation of generating response information based on an analysis of a prompt may be performed by an on-device AI model of the electronic device (201), or by an AI model of an external electronic device, or individually or collectively by the on-device AI model of the electronic device (201) and the AI ​​model of the external electronic device. Although the operation of generating response information has been given as an example only for the convenience of explanation, the on-device AI model of the electronic device (201) and / or the AI ​​model of the external electronic device may be used individually or collectively for other operations requiring the intervention of an AI model.

[0065] In one embodiment, an AI model or a predefined rule of action may be said to be created through learning. Here, being created through learning means that a predefined rule of action or an AI model configured to perform a desired characteristic (or purpose) is created by a basic AI model being trained using a number of training data by a learning algorithm. Such learning may be performed on the device itself (e.g., electronic device (201)) where the AI ​​according to the present disclosure is performed, or it may be performed through a separate server and / or system.

[0066] In one embodiment, the processor (240) may be included in the processor (120) of FIG. 1.

[0067] In one embodiment, the processor (240) may control the overall operation of providing a preview image (and a modified preview image). The processor (240) may include one or more processors for performing the operation of providing a preview image. For example, the processor (240) may correspond to a plurality of processors that collectively perform a plurality of operations by dividing them among the processors. The operation of providing a preview image performed by the processor (240) will be described later.

[0068] In one embodiment, the processor (240) may include a neural processing unit (NPU) for providing a preview image and performing an operation to modify the preview image. For example, the processor (240) may include an NPU capable of performing an operation to modify the preview image using an artificial intelligence model (or AI model) when the operation to modify the preview image is performed using an artificial intelligence model, but is not limited thereto. For example, the processor (240) may include a graphic processing unit (GPU) capable of performing a video acquisition operation using a specified algorithm when the video acquisition operation is performed using a specified algorithm.

[0069] In FIG. 2, the electronic device (201) is illustrated as including a first image sensor (211), a second image sensor (213), a display (220), a memory (230), and / or a processor (240), but is not limited thereto. In one embodiment, the electronic device (201) may further include at least one of the components included in the electronic device (101) of FIG. 1. For example, the processor (240) may further include at least one of an input module (150) (e.g., a microphone) or an acoustic output module (155) (e.g., a speaker). The electronic device (201) may be implemented as a device such as a flexible (e.g., rollable, foldable, multi-foldable) mobile device, a tablet, or a laptop, but is not limited thereto.

[0070] FIG. 3a is a drawing for explaining the location where a camera of an electronic device is placed, according to one embodiment.

[0071] Referring to FIG. 3a, in one embodiment, the front (310) and rear (320) of an electronic device (201) are shown. In the present disclosure, the front (310) may refer to one side of the electronic device (201) including a display (220) (or main display) of the electronic device (201). In the present disclosure, the rear (320) may refer to the opposite side of the front (310) of the electronic device (201). A first image sensor (211) including a plurality of rear cameras (321, 323, 325) may be disposed on the upper rear portion of the electronic device (201). The pixels and / or viewing angles of each of the plurality of rear cameras (321, 323, 325) may differ. A second image sensor (213) (e.g., at least one front camera) may be disposed on the upper portion of the front (310) of the electronic device (201). The second image sensor (213) may be placed, for example, below a hole in the display (220). The second image sensor (213) may also be placed in the center of a notch design. The second image sensor (213) may also be placed outside the display (220). The second image sensor (213) may be placed in an under-display camera (UDC) manner below the display area (or display area) of the display (220).

[0072] FIGS. 3B and FIGS. 3C are drawings for illustrating a method of providing a preview image of an electronic device according to one embodiment.

[0073] Referring to FIG. 3b, in one embodiment, subjects (350, 360) acquired through cameras of an electronic device (201) (e.g., a first image sensor (211) and / or a second image sensor (213)) are shown. The field of view (331) of the first image sensor (211) and the field of view (341) of the second image sensor (213) may be different. For example, the first image sensor (211) (or rear camera) may capture subjects (350) in the direction viewed by the user (361) of the electronic device (201). The electronic device (201) may display (or preview) a preview image acquired through the first image sensor (211) on a display (220). The second image sensor (213) (or front camera) can capture subjects (360) including the face of the user (361) of the electronic device (201). The electronic device (201) can display (or preview) the preview image obtained through the second image sensor (213) on the display (220). The preview image obtained through the first image sensor (211) (e.g., first preview image) and the preview image obtained through the second image sensor (213) (e.g., second preview image) can be displayed, for example, in a picture-in-picture (PIP) manner. In the PIP manner, the second preview image can be displayed small on the first preview image. In one embodiment, the first preview image and the second preview image may be displayed side by side on separate areas (e.g., left area and right area).

[0074] Referring to FIG. 3c, in one embodiment, the electronic device (201) can display a preview image of a multi-view camera. The electronic device (201) can support a multi-view (or multi-preview) mode. In the multi-preview mode, the electronic device (201) can preview an image (e.g., a first preview image (333)) (or image) obtained through a first camera (e.g., a first image sensor (211)) through a display (220), and can preview an image (e.g., a second preview image (343)) obtained through a second camera (e.g., a second image sensor (213)) on the first preview image (333). For example, the first preview image (333) may include images of subjects (350) located in the direction the first camera is looking. The first preview image (333) may be displayed on the first part (371) (or area) of the display (220). The second preview image (343) may include an image of subjects (360) located in the direction the second camera is looking. The second preview image (343) may be displayed on the second part (373) of the display (220). The first camera may be positioned on (or inside) an electronic device such that the direction the first camera is looking is opposite to the direction the second camera is looking, and there are no limitations thereto.

[0075] In one embodiment, the electronic device (201) may generate at least one graphic object (381) based on user input (e.g., at least one of text, speech, or drawing input) for the first preview image (333). The electronic device (201) may generate the graphic object (381) using a machine learning model (e.g., an on-device AI model) trained to generate the graphic object based on receiving a prompt input, for example. The machine learning model may be stored in the electronic device (201). The electronic device (201) may also generate the graphic object (381) using an external electronic device (e.g., electronic device (102) and / or electronic device (104)) or a server (108) connected to the electronic device (201) via communication circuitry (e.g., a communication module (190)). The electronic device (201) may, for example, generate a graphic object (381) representing the sun using a machine learning model (e.g., a generative AI model) based on receiving user input (e.g., a prompt) to display the sun at the top-left corner of the first preview image (333). The electronic device (201) may display the generated object (381) on the first preview image (333). The electronic device (201) may regenerate the first preview image (333) based on the location (or characteristics of the object) where the graphic object (381) representing the sun is displayed on the first preview image (333). The electronic device (201) (or, generative AI model) can modify (or regenerate) the second preview image (343) based on the characteristics of the generated graphic object (e.g., the location of the graphic object, the features of the graphic object, and / or a description associated with the graphic object). The electronic device (201) can apply effects associated with the graphic object (381) to the second preview image (343) based on information associated with the characteristics (or attributes) of the graphic object (381).For example, the brightness of the second preview image (343) may be increased based on the graphic object (381) being displayed on the first preview image (333). In one embodiment, the electronic device (201) may determine the location where the graphic object (381) representing the sun is displayed on the first preview image (333) and the display direction of the second preview image (343) (e.g., information indicating whether it is pivoted). Based on detecting the user's face within the second preview image (343), the electronic device (201) may use a machine learning model to apply a light effect corresponding to the characteristics of the sun (e.g., the position of the sun and / or the direction in which sunlight shines) to the user's face. For example, in the second preview image (343) (or the modified second preview image) with the light effect applied, a bright light effect may be applied to the face located on the side facing the sun, and the area that is shadowed by the sun may be depicted as dark. In one embodiment, when a horizontal flip is applied to the second preview image (343), the electronic device (201) can modify the second preview image (343) based on the characteristics of the graphic object (381) and the display direction of the second preview image (343). When a horizontal flip is applied to the second preview image (343), an effect such as a graphic object (381) representing a sun being positioned at the upper right corner of the first preview image (333) can be applied to the second preview image (343), although not shown in FIG. 3c.

[0076] In one embodiment, an effect applied to a preview image (e.g., “AI effect” or “AI object-based effect”) may be disabled based on whether an AI object is located within the preview image. For example, if a graphic object (381) moves out of the field of view of the first image sensor (e.g., the field of view (331) in FIG. 3b) due to movement of the electronic device (201), a shadow effect applied to the second preview image (343) may be disabled. For example, the electronic device (201) may refrain from displaying the modified second preview image. The electronic device (201) may display the second preview image without the shadow effect applied. In one embodiment, the electronic device (201) may maintain the shadow effect applied to the second preview image (343) even if the graphic object (381) moves out of the field of view of the first image sensor. For example, the electronic device (201) may determine (or verify) the characteristics of the graphic object (381). The electronic device (201) can determine that the characteristics of the graphic object (381) are predefined characteristics or characteristics that are predicted to continuously affect the second preview image (343). Based on the characteristics of the identified graphic object (381), the electronic device (201) can apply effects to the first preview image (333) and / or the second preview image (343) even if the graphic object (381) is out of the field of view of the first image sensor. The electronic device (201) can apply shadow effects differently to the first preview image (333) and the second preview image (343), respectively, based on the direction in which the graphic object (381) (e.g., the sun) moved (or the trajectory outside the field of view).

[0077] In one embodiment, the electronic device (201) may capture a first preview image (333) and a second preview image (343) with effects based on the characteristics of a graphic object (381) applied, based on user input (e.g., touch input) to an object (not shown) for capturing a preview image. When the captured first preview image (333) and the captured second preview image (343) are stored, information associated with the effects applied to the graphic object (381) and / or the preview images may be stored as metadata. In one embodiment, the electronic device (201) may sequentially store the first preview image (333) and the second preview image (343), respectively, based on sequential user input. For example, the electronic device (201) may capture the first preview image (333) based on user input for capturing the first preview image (333) in which a graphic object (381) is displayed, and may store information associated with the characteristics of the first preview image (333) and the graphic object (381). In one embodiment, based on the storage of the first preview image (333), an effect corresponding to the graphic object (381) (e.g., an AI shadow effect) may be applied to the second preview image (343) displayed through the display (220). While the second preview image (343) is displayed through the display (220) (or until user input for capturing the second preview image (343) is received), the stored first preview image (333) may be displayed as a still image. The electronic device (201) can take a second preview image (343) based on user input for taking a second preview image (343), and store information related to the second preview image (343) and the effect applied to the second preview image (343).In one embodiment, a second preview image (343) to which an effect corresponding to a graphic object (381) has not been applied may be displayed through the display (220). The capture of preview images based on sequential user input may be performed based on the operation of a timer. For example, the timer may be executed (or started) from the time when the first preview image is captured. Until the timer expires, the user of the electronic device (201) may adjust the direction (or angle of view) of the second image sensor (213). Based on the expiration of the timer, the second preview image (343) may be captured.

[0078] In one embodiment, the first preview image (333) and the second preview image (343) may be stored in memory (e.g., memory (230)). The first preview image (333) and the second preview image (343) may be stored as a single image having a PIP format. Each of the first preview image (333) and the second preview image (343) may be stored as a different image (or image file). For example, information associated with an effect corresponding to a graphic object (381) may be stored in each of the first preview image (333) and the second preview image (343). Along with the first preview image (333), information of a graphic object representing the sun may be stored. Along with the second preview image (343), information associated with a shading effect caused by the sun at a specified location may be stored.

[0079] In one embodiment, in FIGS. 3a, 3b, and 3c, the electronic device (201) is depicted as a bar-type device for convenience of explanation, but is not limited thereto. For example, the application of effects to graphic objects in a multi-view camera as described in FIGS. 3a, 3b, and 3c may also be applied in an XR (extended reality) environment. When a user of the electronic device (201) applies a specific effect to a VR screen acquired through a camera while wearing a VR (virtual reality) device, the specific effect (or other effects resulting from the influence of the specific effect) may also be applied to other virtual objects. For example, a user of a VR device may draw an AI object (e.g., the sun) on a previewed image through a camera while wearing a VR device. An effect (e.g., a shadow or shading effect) corresponding to the characteristics (or location) of the AI ​​object displayed by the user may be applied to an icon (or auxiliary screen) overlaid on the preview image. The electronic device (201) can provide a vivid and seamless multi-preview image by applying effects associated with the graphic object (381) to the second preview image (343) beyond simply displaying the graphic object (381) on the first preview image (333).

[0080] FIG. 4 is a drawing for explaining a method of providing a modified preview image of an electronic device according to one embodiment.

[0081] Referring to FIG. 4, in one embodiment, the AI ​​model (400) can output a modified first preview image (431) and a modified second preview image (433) based on receiving a first preview image (333) obtained through a first image sensor (e.g., first image sensor (211)) and a second preview image (343) obtained through a second image sensor (e.g., second image sensor (213)). The AI ​​model (400) may include, for example, a first AI model (410) and a second AI model (420). In one embodiment, the AI ​​model (400) may be composed of a plurality of neural network layers. Each of the plurality of neural network layers has a plurality of weight values ​​and can perform neural network operations through operations between the operation result of a previous layer and the plurality of weights. Multiple weights of multiple neural network layers can be optimized based on the learning results of the AI ​​model (400). For example, multiple weights can be updated so that a loss value or cost value obtained from the AI ​​model (400) during the learning process is reduced or minimized. In one embodiment, as the operation (or process) of updating the weights is repeated, the inference performance of the AI ​​model (400) can be improved.

[0082] In one embodiment, the first AI model (410) may be trained to recognize objects, classify objects, estimate depth information, and analyze scenes based on receiving the first preview image (333) and the second preview image (343) as input. The object recognition and classification model (411) may recognize objects included in the input images and classify each of the recognized objects. For object recognition and / or classification, for example, a segmentation AI model may be used. The segmentation AI model may identify (or distinguish) specific objects or regions based on a single input image and output information associated with the objects or regions. The depth information estimation model (413) may estimate the distance between objects (or backgrounds) and the captured scene based on image data (e.g., 2D data). When applying a stereo matching algorithm, which is another method for estimating distance, the time complexity may increase. The depth information estimation model (413) can provide (or generate) a preview image (or AI-based preview image) quickly and naturally by estimating depth information from a single image input in real time. The scene analysis model (415) can output information related to the environment (or atmosphere) in which the image was taken based on the analysis of the input image. The scene analysis model (415) can be trained using a pair of training data containing input data (e.g., a single image) and ground truth (e.g., context information) corresponding to the input data. The scene analysis model (415) can output a natural AI preview image based on the analysis of the preview image and the objects included in the preview image.

[0083] In one embodiment, the second AI model (420) may include a generative AI model trained to output a modified first preview image (431) and a modified second preview image (433) based on receiving the output of the first AI model (410) and user input (or prompt). User input may include at least one of text, speech, or drawing input, as described above with reference to FIG. 3c. The second AI model (420) may be used to generate graphic objects. The second AI model (420) may generate graphic objects based on user input. The generative AI model may transform the shape of an object, for example. The generative AI model may be trained using a dataset (or training data pair) that includes the initial state of the object to be transformed and the object to be transformed (or ground truth). The generative AI model can perform detailed training (or fine-tuning) based on the output information of the depth information estimation model (413) and the scene analysis model (415). The generative AI model can transform the shape of an object based on training and fine-tuning using a dataset. For example, the generative AI model can output a transformation target object based on the transformation target object and user input. Because the generative AI model is trained using the depth information estimation model (413), it can pre-specify the limits of the transformation target object (or the maximum range of transformation). The generative AI model can edit a preview image containing the transformation target object by considering the correlation between the transformation target object and other objects (or physical relationships identified within the image) based on depth information. The generative AI model can also generate additional objects when the transformation target object is acquired by using the scene analysis model (415).

[0084] In one embodiment, the second AI model (420) can transform the shape of a winter tree into a cherry tree (421) based on receiving output information from the first AI model (410) and user input regarding a first preview image (333) containing a winter tree. The second AI model (420) can input the transformed object (e.g., characteristic information of the cherry tree (421)) to the first AI model (410). The first AI model (410) can provide the second AI model (420) with object recognition results, object classification results, depth information estimation results, and / or scene analysis results identified from the characteristic information of the cherry tree (421) and the second preview image (343), respectively. The second AI model (420) can modify (or regenerate) the second preview image (343) based on the output information from the first AI model (410) and user input. For example, if a user in winter attire is identified within the second preview image (343), the second AI model (420) can output a modified second preview image (433) including a user's face in spring attire by reflecting the characteristics of a cherry tree (421). The second AI model (420) can provide a modified first preview image (431) including a modified object (421) and a modified second preview image (433) based on the modified object (421) in a PIP format.

[0085] FIG. 5 is a flowchart illustrating a method for providing a modified preview image of an electronic device according to one embodiment. The embodiment of FIG. 5 is described with reference to FIG. 6a, 6b, 6c, 6d, 7a, and 7b. FIG. 6a, 6b, 6c, and 6d are drawings illustrating a method for providing a modified preview image of an electronic device according to one embodiment. FIG. 7a and 7b are drawings illustrating a method for providing modified image frames of an electronic device according to one embodiment.

[0086] In one embodiment, the operations illustrated in FIG. 5 may be performed in various orders, not limited to the order illustrated. For example, the order of each operation may be changed, and at least two operations may be performed in parallel. According to one embodiment, more operations may be performed than those illustrated in FIG. 5, or at least one fewer operation may be performed.

[0087] Referring to FIG. 5, in operation 501, in one embodiment, an electronic device (201) (e.g., processor (240)) can acquire a first preview image. The electronic device (201) can acquire the first preview image, for example, through a first image sensor (e.g., first image sensor (211)).

[0088] In operation 503, in one embodiment, the electronic device (201) can acquire a second preview image. The electronic device (201) can acquire the second preview image, for example, through a second image sensor (e.g., a second image sensor (213)).

[0089] In operation 505, in one embodiment, the electronic device (201) may display a first preview image on a first part (or first area) of the touchscreen display and a second preview image on a second part (or second area) of the touchscreen display through a touchscreen display (e.g., display (220)). In one embodiment, the way the preview images are displayed on the first part and the second part may be based on a PIP method. For example, on the touchscreen display, another preview image may be displayed small on top of one preview image using a PIP method. In one embodiment, two preview images may be displayed side by side (e.g., left and right) on separate areas.

[0090] Referring to FIG. 6a, in one embodiment, the electronic device (201) may display the first preview image (610) and the second preview image (343) based on overlaying the second preview image (343) on the first preview image (610). The electronic device (201) may display the first preview image (610) on a first portion (371) of the display (220) and the second preview image (343) on a second portion (373) of the display (220) through the display (220). The first preview image (610) may include an image of a subject (601) (e.g., a winter tree). The second preview image (343) may include an image of a user's face facing the second image sensor (213).

[0091] In operation 507, in one embodiment, the electronic device (201) may receive user input for creating an AI object for a first preview image (610). The user input may include, for example, a drawing input, but is not limited thereto. The user input may be at least one of text, speech, or drawing input. The user input may be a combination of at least two of text, speech, or drawing input.

[0092] In operation 509, in one embodiment, the electronic device (201) may generate an AI object using a machine learning model (e.g., an AI model (400)) based on user input. The machine learning model may include a generative AI model trained to output at least one AI object based on text, voice, or drawing input. For example, the electronic device (201) may display at least one object corresponding to the drawing input on the first preview image (610). The electronic device (201) may generate an AI object based on at least one object.

[0093] Referring to FIG. 6b, in one embodiment, the electronic device (201) can display at least one object (611) corresponding to a drawing input on the first preview image (610). For example, the electronic device (201) (e.g., the object recognition and classification model (411) of FIG. 4) can generate at least one object (611) based on an input of painting pink paint on a tree.

[0094] Referring to FIG. 6c, in one embodiment, the electronic device (201) can generate an AI object (621) based on at least one generated object (611). The electronic device (201) can generate an AI object (621) representing, for example, a cherry tree. The electronic device (201) can display a first preview image (620) containing the AI ​​object (621) through a display (220).

[0095] In operation 511, in one embodiment, the electronic device (201) can modify the first preview image and the second preview image using a machine learning model based on the characteristics of the generated AI object. For example, if the AI ​​object generated based on user input represents a cherry tree, the electronic device (201) can modify the second preview image based on the characteristics of the cherry tree. Referring to FIG. 7a, the AI ​​model (400) can generate the modified first image frames (740) and the modified second image frames (750), respectively, in real time based on receiving image frames (710) sequentially acquired through a first image sensor (e.g., first image sensor (211)), image frames (720) sequentially acquired through a second image sensor (e.g., second image sensor (213)), and a prompt (730). The AI ​​model (400) can provide the modified first image frames (740) as the first preview image. The AI ​​model (400) can provide the modified second image frames (750) as the second preview image. Referring to FIG. 7b, the AI ​​model (400) may generate modified preview image frames (760) based on receiving image frames (710) sequentially acquired through the first image sensor, image frames (720) sequentially acquired through the second image sensor, and a prompt (730). Each of the modified preview image frames (760) may be a PIP-type image frame containing the modified first preview image and the modified second preview image. The AI ​​model (400) can provide the modified preview image frames (760) as the modified preview image.

[0096] In operation 513, in one embodiment, the electronic device (201) may display a modified first preview image in a first part and a modified second preview image in a second part. Referring to FIG. 6d, the electronic device (201) may display a modified first preview image (630) in a first part (371) and a modified second preview image (640) in a second part (373). The electronic device (201) may display the modified first preview image (630) and the modified second preview image (640) through the display (220) by modifying the user's form (or attire) included in the second preview image (343) based on the characteristics of the AI ​​object (631) representing a cherry tree.

[0097] In one embodiment, the electronic device (201) may first apply an effect corresponding to an AI object to a first preview image (610) as shown in FIGS. 6a and 6b, and then sequentially apply an effect corresponding to an AI object to a second preview image (343). For example, the electronic device (201) may display a first preview image (620) modified based on a modified AI object (621) in the first part (371) based on user input. After displaying the modified first preview image (620), the electronic device (201) may display a second preview image (640) modified based on an AI object (621) in the second part (373). The electronic device (201) may, for example, acquire a modified second preview image (640) using information associated with the characteristics of the AI ​​object (621) and the second preview image (343) based on receiving user input for displaying a modified second preview image (640). In one embodiment, the application of an AI effect to the preview image may be performed using a stored multi-view captured image in addition to the preview image acquired in real time.

[0098] In one embodiment, the electronic device (201) may receive other user input for capturing at least a portion of the modified first preview image (630) and the modified second preview image (640). Based on the other user input, the electronic device (201) may store a captured image including at least a portion of the modified first preview image (630) and the modified second preview image (640). The modified first preview image (630) and the modified second preview image (640) may be stored as a single image file having a PIP format. Each of the modified first preview image (630) and the modified second preview image (640) may also be stored as a different image file. The electronic device (201) may store metadata associated with the AI ​​object (631) together with the captured image.

[0099] In one embodiment, the position where the AI ​​object (631) is displayed on the first preview image (630) may be changed based on the movement of the electronic device (201). For example, the AI ​​object (631) may be displayed at different positions on the first preview image (630) based on its relative position within the field of view of a first image sensor (e.g., the first image sensor (211)) placed on the electronic device (201). The AI ​​object (631) may move out of the field of view of the first image sensor due to a sudden movement of the electronic device (201). If the AI ​​object (631) moves out of the first preview image (630) based on the movement of the electronic device (201), the electronic device (201) may display a second preview image without the AI ​​object (631) (e.g., a second preview image in which the application of effects based on the characteristics of the AI ​​object (631) is released).

[0100] In one embodiment, the first preview image (333) and the second preview image (343) may each be a preview image obtained by a different electronic device. For example, the electronic device (201) may establish a communication connection with an external electronic device (e.g., electronic device (102), electronic device (104), and / or server (108)) through a communication circuit (e.g., communication module (190) of FIG. 1). The electronic device (201) and the external electronic device may be functionally connected. The connection (or sharing) function of the preview images may be provided based on the execution of a camera application (e.g., multi-camera function) or a video call. The electronic device (201) may display the first preview image (333) obtained through a first image sensor (e.g., first image sensor (211)) and the second preview image (343) obtained by an external electronic device in a PIP format through a display (220). Each preview image may be displayed in parallel on separate areas on the left and right. When the first preview image (333) and the second preview image (343) are displayed in parallel, the first preview image (333) may be displayed larger than the second preview image (343). The external electronic device may display the first preview image (333) obtained by the electronic device (201) and the second preview image obtained by the external electronic device through the display of the external electronic device. In one embodiment, the electronic device (201) may display an AI object (621) on the first preview image (333) based on user input. The second preview image (343) provided by the external electronic device may be modified based on the characteristics of the AI ​​object (621). The electronic device (201) and the external electronic device can each display a modified second preview image (640) containing an AI object generated in response to the characteristics of the AI ​​object (621).In one embodiment, the electronic device (201) can generate an AI object of a rainy landscape on a first preview image (610) based on user input. On the second preview image (640), an effect corresponding to the characteristics of the AI ​​object generated on the first preview image (610) may be applied. For example, a rain effect may be applied on the second preview image (640). In one embodiment, on the second preview image (640), an effect corresponding to the characteristics of the AI ​​object generated on the first preview image (610) may be applied based on sensor information (e.g., location information) obtained by an external electronic device. For example, the electronic device (201) can use GPS information from the external electronic device to determine that the region where the external electronic device is located is a cold region. The electronic device (201) can generate a second preview image with a snow effect applied based on information associated with the characteristics of the generated AI object and GPS information from the external electronic device.

[0101] In one embodiment, the electronic device (201) can enhance the user experience by displaying a first preview image (630) modified based on an AI object (631) and a second preview image (640) modified based on an AI object (631).

[0102] FIG. 8 is a flowchart illustrating a method for providing a modified preview image of an electronic device according to one embodiment. The embodiment of FIG. 8 will be described with reference to FIG. 9. FIG. 9 is a diagram illustrating a method for providing a modified preview image of an electronic device according to one embodiment.

[0103] In one embodiment, the operations illustrated in FIG. 8 may be performed in various orders, not limited to the order illustrated. For example, the order of each operation may be changed, and at least two operations may be performed in parallel. According to one embodiment, more operations may be performed than those illustrated in FIG. 8, or at least one fewer operation may be performed.

[0104] Referring to FIG. 8, in operation 801, in one embodiment, an electronic device (201) (e.g., processor (240)) may display a first preview image and a second preview image through a display (e.g., display (200)). The electronic device (201) may display, for example, a first preview image obtained through a first image sensor (e.g., first image sensor (211)) and a second preview image obtained through a second image sensor (e.g., second image sensor (213)) through a display (e.g., display (220)).

[0105] Referring to FIG. 9, in one embodiment, the process of transforming a first preview image (910) and a second preview image (920) by an electronic device (201) is illustrated. The first preview image (910) may include, for example, an image of a tree (911) with only branches. The second preview image (920) may include, for example, a face (921) of a user wearing winter clothes.

[0106] In operation 803, in one embodiment, the electronic device (201) can generate an AI object using a machine learning model. Referring again to FIG. 9, the electronic device (201) can generate (912) a cherry tree (913) using a machine learning model (e.g., an AI model (400)) based on user input. The electronic device (201) can modify (922) a second preview image (920) based on the characteristics of the generated cherry tree (913). For example, the modified second preview image may include the face (923) of a user wearing spring clothes. In operation 805, in one embodiment, the electronic device (201) can display the modified first preview image and the modified second preview image through a display.

[0107] In operation 807, in one embodiment, the electronic device (201) may receive other user input regarding the modified second preview image. Referring to FIG. 9, the electronic device (201) may receive user input for creating an object representing a street light (925). The electronic device (201) may receive user input for modifying at least one object of the second preview image while the modified second preview image is displayed based on the AI ​​object displayed in the first preview image.

[0108] In operation 809, in one embodiment, the electronic device (201) may modify the modified second preview image using a machine learning model. The electronic device (201) may generate other AI objects based on other user inputs. Referring to FIG. 9, the electronic device (201) may generate (924) a modified second preview image including a street light (925) shining light over the user's head based on user input (or a prompt) for generating an AI object representing a street light. The electronic device (201) may display the modified second preview image based on other AI objects. The modified second preview image may include other AI objects. Effects based on the characteristics (e.g., spring) of the AI ​​object (e.g., cherry tree (915) in FIG. 9) may be applied to the modified second preview image.

[0109] In operation 811, in one embodiment, the electronic device (201) can modify a modified first preview image using a machine learning model. The electronic device (201) can modify the first preview image (914) based on the modified second preview image. For example, the electronic device (201) can apply a brightness effect to a cherry tree (915). A first image sensor (211) may be placed on a first surface of the electronic device (201), and a second image sensor (213) and a display may be placed on a second surface opposite to the first surface. The electronic device (201) can modify the first preview image based at least on the position where the first image sensor is placed on the electronic device. For example, the electronic device (201) can identify the area of ​​the cherry tree (915) that needs to be brightened by a street light (925) based on the relative position between the first image sensor and the second image sensor. The electronic device (201) can increase the brightness of a portion of the cherry tree (915) near the street light (925) and apply a darkening effect to the area that is shadowed by the light of the street light (925). The electronic device (201) can display a first preview image modified based on the AI ​​object and the other AI object based on the other AI object.

[0110] FIG. 10 is a flowchart illustrating a method for providing a modified preview image of an electronic device according to one embodiment.

[0111] In one embodiment, the operations illustrated in FIG. 10 may be performed in various orders, not limited to the order illustrated. For example, the order of each operation may be changed, and at least two operations may be performed in parallel. According to one embodiment, more operations may be performed than those illustrated in FIG. 10, or at least one fewer operation may be performed.

[0112] Referring to FIG. 10, in operation 1001, in one embodiment, an electronic device (201) (e.g., processor (240)) can acquire a first preview image. The electronic device (201) can acquire the first preview image through a first image sensor (e.g., first image sensor (211)).

[0113] In operation 1003, in one embodiment, the electronic device (201) can acquire a second preview image. The electronic device (201) can acquire the second preview image through a second image sensor (e.g., a second image sensor (213)).

[0114] In operation 1005, in one embodiment, the electronic device (201) can display a first preview image on a first part (or first area) of the touchscreen display and a second preview image on a second part (or second area) of the touchscreen display through a touchscreen display (e.g., display (220)).

[0115] In operation 1007, in one embodiment, the electronic device (201) may obtain a modified first preview image containing a first object. The electronic device (201) may input the first preview image to a generative AI model (e.g., AI model (400)) based on receiving user input for displaying the first object (e.g., graphic object (381) of FIG. 3c) on the first preview image. The generative AI model may be a model trained to display the graphic object on the input preview image based on receiving the preview image. The electronic device (201) may obtain a modified first preview image containing the first object based on information output from the generative AI model.

[0116] In operation 1009, in one embodiment, the electronic device (201) can obtain a modified second preview image based on information output from the generative AI model by inputting information associated with the characteristics of the first object and the second preview image into the generative AI. The electronic device (201) can obtain a modified first preview image and a modified second preview image based on receiving user input for displaying the first object on the first preview image without sequential user input.

[0117] In operation 1011, in one embodiment, the electronic device (201) may display a modified first preview image in a first part and a modified second preview image in a second part through a touchscreen display. The electronic device (201) may perform a non-discreet modification between the preview images by regenerating the first preview image based on a first object created on the first preview image and / or regenerating the second preview image based on the first object.

[0118] FIG. 11 is a flowchart illustrating a method for providing a modified preview image of an electronic device according to one embodiment.

[0119] In one embodiment, the operations illustrated in FIG. 11 may be performed in various orders, not limited to the order illustrated. For example, the order of each operation may be changed, and at least two operations may be performed in parallel. According to one embodiment, more operations may be performed than those illustrated in FIG. 11, or at least one fewer operation may be performed.

[0120] Referring to FIG. 11, in operation 1101, in one embodiment, an electronic device (201) (e.g., processor (240)) can acquire a first preview image. The electronic device (201) can acquire the first preview image through a first image sensor (e.g., first image sensor (211)).

[0121] In operation 1103, in one embodiment, the electronic device (201) can acquire a second preview image. The electronic device (201) can acquire the second preview image through a second image sensor (e.g., a second image sensor (213)).

[0122] In operation 1105, in one embodiment, the electronic device (201) can display a first preview image on a first part (or first area) of the touchscreen display and a second preview image on a second part (or second area) of the touchscreen display through a touchscreen display (e.g., display (220)).

[0123] In operation 1107, in one embodiment, the electronic device (201) may receive user input for creating a graphic object (e.g., the graphic object (381) of FIG. 3c) on a first preview image. For example, the electronic device (201) may receive user input based on an object (or function) for creating the graphic object. The graphic object that can be created may be set in memory (e.g., memory (230)). The electronic device (201) may include an AI model for creating the graphic object.

[0124] In operation 1109, in one embodiment, the electronic device (201) may create a graphic object on the first preview image. The electronic device (201) may create the graphic object by, for example, based on a rule-based algorithm, or by using an AI model trained to create the graphic object based on receiving a prompt.

[0125] In operation 1111, in one embodiment, the electronic device (201) may display a modified first preview image containing a graphic object in a first part of a touchscreen display based on at least some of the characteristics of the graphic object.

[0126] In operation 1113, in one embodiment, the electronic device (201) can modify the second preview image. The electronic device (201) can modify the second preview image based on sequential user input after the modified first preview image has been displayed. The electronic device (201) can modify the second preview image by applying an effect corresponding to the characteristics of the graphic object to the second preview image based on the characteristics of the generated graphic object.

[0127] In operation 1115, in one embodiment, the electronic device (201) may display a second preview image modified in a second part. The electronic device (201) may enhance the user experience by displaying a second preview image modified based on the modification of the first preview image while the multi-view function is being executed.

[0128] FIG. 12 is a flowchart illustrating a method for providing a modified preview image of a foldable electronic device according to one embodiment. The embodiment of FIG. 12 is described with reference to FIG. 13a, FIG. 13b, FIG. 13c, FIG. 13d, and FIG. 13e. FIG. 13a is a drawing illustrating a foldable electronic device according to one embodiment. FIG. 13b is a drawing illustrating a method for providing a preview image of a foldable electronic device according to one embodiment. FIG. 13c, FIG. 13d, and FIG. 13e are drawings illustrating a method for providing a modified preview image of a foldable electronic device according to one embodiment.

[0129] In one embodiment, the operations illustrated in FIG. 12 may be performed in various orders, not limited to the order illustrated. For example, the order of each operation may be changed, and at least two operations may be performed in parallel. According to one embodiment, more operations may be performed than those illustrated in FIG. 12, or at least one fewer operation may be performed.

[0130] Referring to FIG. 12, in operation 1201, in one embodiment, an electronic device (201) (e.g., processor (240)) can acquire a first preview image through a first image sensor disposed in a first housing on a first side of the foldable electronic device. The electronic device (201) may include, for example, at least one hinge (or hinge portion). The electronic device (201) may have different folding states based on the state of the hinge (e.g., fully folded state, partially folded state, or fully unfolded state). If the electronic device (201) includes two or more hinges, it may be referred to as a "multi-foldable device."

[0131] Referring to FIG. 13a, the electronic device (201) can be implemented as an e-type electronic device (1311), a G-type electronic device (1321), or a z-type electronic device (1331).

[0132] Referring to reference numeral 1310, the folded or unfolded state of the e-type electronic device (1311) is shown.

[0133] Referring to reference numeral 1313, a fully folded state of an e-type electronic device (1311) is illustrated. The e-type electronic device (1311) may include a sub-display (1301) and a second image sensor (213) formed on at least a portion of the sub-display (1301). The fully folded state of the e-type electronic device (1311) may correspond to a state in which the hinges (1303a, 1303b) of the e-type electronic device (1311) are fully folded.

[0134] Referring to reference numeral 1315, a surface including the main display (1302) (or flexible display) of the e-type electronic device (1311) is shown in a fully unfolded state of the e-type electronic device (1311). The main display (1302) of the e-type electronic device (1311) may be divided into three regions based on the hinges (1303a, 1303b). The main display (1302) may include, for example, a first housing, a second housing, and a third housing. In one embodiment, a third image sensor (not shown) may be formed on at least a portion of the main display (1302). Referring to reference numeral 1317, a surface including the sub-display (1301) of the e-type electronic device (1311) is shown in a fully unfolded state of the e-type electronic device (1311). A sub-display (1301) may be placed on the outer surface of a main display (e.g., the main display (1302) of reference numeral 1315). The main display (1302) may include a first housing (1304b), a second housing (1304a), and a third housing (1304c). A sub-display (1301) may be placed on the outer surface of the second housing (1304a). An e-type electronic device (1311) may include a first hinge (1303a) that rotatably connects the first housing (1304b) and the second housing (1304a), and a second hinge (1303b) that rotatably connects the second housing (1304a) and the third housing (1304c). The second hinge (1303b) connecting the second housing (1304a) and the third housing (1304c) may have a width smaller than the first hinge (1303a) connecting the first housing (1304b) and the second housing (1304a). The sub-display (1301) may be supported, for example, by the second housing (1304a). The first image sensor (211) may be placed on the outer surface of the first housing (1304b).The first image sensor (211) may include a plurality of rear cameras spaced apart by a first distance (1305). The first image sensor (211) may be positioned at a location spaced apart by a second distance (1306) from the second image sensor (213). In the fully folded state of the e-type electronic device (1311), the first image sensor (211) may be positioned to face in the opposite direction to the second image sensor (213). In the fully unfolded state of the e-type electronic device (1311), the first image sensor (211) may be positioned to face in the same direction as the second image sensor (213).

[0135] Referring to reference numeral 1320, the folded or unfolded state of the G-type electronic device (1321) is shown.

[0136] Referring to reference numeral 1323, a fully folded state of a G-type electronic device (1321) is illustrated. The G-type electronic device (1321) may include a sub-display (1301) and a second image sensor (213) formed on at least a portion of the sub-display (1301). The fully folded state of the G-type electronic device (1321) may correspond to a state in which the hinges (1303a, 1303b) of the G-type electronic device (1321) are fully folded.

[0137] Referring to reference numeral 1325, a surface including the main display (1302) of the G-type electronic device (1321) in a fully unfolded state is shown. The main display (1302) of the G-type electronic device (1321) may be divided into three regions based on hinges (1303a, 1303b). The main display (1302) may include, for example, a first housing, a second housing, and a third housing. A third image sensor (not shown) may be disposed on the top of the main display (1302). Referring to reference numeral 1327, a surface including the sub-display (1301) of the G-type electronic device (1321) in a fully unfolded state is shown. A sub-display (1301) may be placed on the outer surface of a main display (e.g., the main display (1302) of reference numeral 1325). The main display (1302) may include a first housing (1304b), a second housing (1304a), and a third housing (1304c). The sub-display (1301) may be placed on the outer surface of the third housing (1304c). The sub-display (1301) may be supported, for example, by the third housing (1304c). A first image sensor (211) may be placed on the outer surface of the second housing (1304a). The first image sensor (211) may include a plurality of rear cameras spaced apart by a first distance (1305). The first image sensor (211) can be positioned at a distance (1306) from the second image sensor (213).

[0138] Referring to reference numeral 1330, a z-type electronic device (1331) in a folded or unfolded state is shown.

[0139] Referring to reference numeral 1333, a fully folded state of a z-type electronic device (1331) is illustrated. The z-type electronic device (1331) may include a second image sensor (213) formed on at least a portion of the main display (1302). The main display (1302) may include a first housing, a second housing, and a third housing. The second image sensor (213) may be included in, for example, the third housing (e.g., the third housing (1304c) of reference numeral 1337 described later). The fully folded state of the z-type electronic device (1331) may correspond to a state where the hinges (1303a, 1303b) of the z-type electronic device (1331) are fully folded.

[0140] Referring to reference numeral 1335, a surface (e.g., front) including the main display (1302) of the z-type electronic device (1331) is shown in a fully unfolded state of the z-type electronic device (1331). The main display (1302) of the z-type electronic device (1331) may be divided into three regions based on the hinges (1303a, 1303b). Referring to reference numeral 1337, a rear surface of the z-type electronic device (1331) is shown in a fully unfolded state of the z-type electronic device (1331). The rear surface of the z-type electronic device (1331) may include the outer surface of the first housing (1304b), the outer surface of the second housing (1304a), and the outer surface of the third housing (1304c). A first image sensor (211) may be placed on the outer surface of the first housing (1304b). The first image sensor (211) may include a plurality of rear cameras.

[0141] In operation 1203, in one embodiment, the electronic device (201) can acquire a second preview image through a second image sensor disposed in a second housing on a first side of the foldable electronic device.

[0142] In operation 1205, in one embodiment, the electronic device (201) may display a first preview image in a first part of the first display and a second preview image in a second part of the first display.

[0143] Referring to FIG. 13b, in one embodiment, a preview image corresponding to the folding state of an e-type electronic device (1311) is shown.

[0144] Referring to reference numeral 1313, in a fully folded state of the e-type electronic device (1311), the second image sensor (213) of the e-type electronic device (1311) may be positioned on the opposite side (or facing in the opposite direction) of the first image sensor (e.g., the first image sensor (211)). The fully folded state may refer to a state in which both the first hinge (e.g., the first hinge (1303a) of FIG. 13a) and the second hinge (e.g., the second hinge (1303b) of FIG. 13a)) are fully folded. The first image sensor (211) may capture a background (350) located opposite the user. The second image sensor (213) may capture subjects (360) located in the direction of the user's face. The direction from the e-type electronic device (1311) toward the background (350) may be referred to as the "first direction." The opposite direction of the first direction may be referred to as the "second direction." The second direction may be a direction toward the user (or subjects (360) located toward the user's face). The e-type electronic device (1311) may display both preview images (first preview image and second preview image) simultaneously. The e-type electronic device (1311) may display the first preview image (333) and the second preview image (343) in the sub-display (1301) area. The first preview image (333) may be a preview image in which the subject of the first direction is captured. The second preview image (343) may be a preview image in which the subject of the second direction is captured. The e-type electronic device (1311) may display only a preview image obtained through a front camera (e.g., a second image sensor (213)) in the sub-display (1301) area, or display only a preview image obtained through a rear camera (e.g., a first image sensor) in the sub-display (1301) area.When the electronic device (201) is in a fully folded state, it can acquire a first preview image in which a subject in a first direction is captured through a first image sensor (211), and acquire a second preview image in which a subject in a second direction (360) is captured through a second image sensor (213).

[0145] Referring to reference numeral 1341, in a partially unfolded state (e.g., once unfolded) of the e-type electronic device (1311), the second image sensor of the e-type electronic device (1311) may be positioned to face in the same direction as the first image sensor (e.g., toward the background (350)). The partially unfolded state may refer to a state where the first hinge is fully unfolded and the second hinge is fully folded. The first image sensor and the second image sensor may capture the background (350) located opposite the user. The field of view of the first image sensor may be different from the field of view of the second image sensor. The e-type electronic device (1311) may simultaneously display both preview images in a portion of the right side of the main display (1302). The e-type electronic device (1311) can display, for example, a preview image (1343a) obtained through a first image sensor and a preview image (1343b) obtained through a second image sensor. The e-type electronic device (1311) may display only the preview image obtained through a front camera (e.g., the second image sensor (213)) in the main display (1302) area, or display only the preview image obtained through a rear camera (e.g., the first image sensor (211)) in the main display (1302) area. The e-type electronic device (1311) displays a preview image (e.g., a second preview image (343)) obtained by a front camera (e.g., a second image sensor (213)) on a sub-display (1301) when fully folded, and may also display a preview image (1343a) obtained by a rear camera (e.g., a first image sensor (211)) on a right portion of the main display (1302) based on one hinge being unfolded.

[0146] Referring to reference numeral 1315, a preview image can be displayed over the entire area of ​​the main display (1302) in a fully unfolded state of the e-type electronic device (1311). The fully unfolded state may refer to a state in which both the first hinge and the second hinge are fully unfolded. The e-type electronic device (1311) can display a first preview image (333) obtained through a rear camera having a wide field of view (e.g., a wide-angle camera included in the first image sensor (211)) in a fully folded state, for example. The e-type electronic device (1311) can display a second preview image (343) obtained through a front camera (e.g., a second image sensor (213)) in a fully folded state on an area of ​​the sub-display (1301), display a first preview image (1343a) obtained through a rear camera with a narrow field of view (e.g., a telephoto camera included in the first image sensor (211)) in a partially unfolded state on a right portion of the main display (1302), and display a preview image (1345) obtained through a wide-angle camera in a fully unfolded state on the entire area of ​​the main display (1302). The electronic device (201) can obtain a preview image (1345) in which a subject in a first direction (e.g., a direction facing the background (350)) is captured through the first image sensor (211) and / or the second image sensor (213) in an unfolded state.

[0147] Referring to reference numeral 1343, in a partially folded state (e.g., a once-folded state) of the e-type electronic device (1311), the e-type electronic device (1311) may display a preview image (1347a) obtained through the first image sensor (211) and a preview image (1347b) obtained through the second image sensor (213) on a left portion of the main display (1302). The partially folded state may refer to a state where the first hinge is completely folded and the second hinge is completely unfolded. The field of view of the rear cameras (323, 325) included in the first image sensor (211) may differ. The e-type electronic device (1311) may display only the preview image (1347a) containing an image of the user's face on a left portion of the main display (1302). The e-type electronic device (1311) may display only a preview image (1347b) containing a background image on a left portion of the main display (1302). The electronic device (201), in a partially folded state, may acquire a preview image (1347a) in which a subject facing the user is captured through the first image sensor (211), and acquire a preview image (1347b) in which a subject facing the background is captured through the second image sensor (213). The embodiment of FIG. 13b is intended to exemplarily explain that the direction in which the first image sensor (211) and / or the second image sensor (213) face may be changed differently depending on the folded state of the electronic device (201) (e.g., e-type electronic device (1311)), and the preview image is not limited to a specific image.

[0148] In operation 1207, in one embodiment, the electronic device (201) may receive user input for creating at least one graphic object for a first preview image.

[0149] In operation 1209, in one embodiment, the electronic device (201) can create a graphic object on the first preview image.

[0150] In operation 1211, in one embodiment, the electronic device (201) may display a modified first preview image containing a graphic object in a first part of the first display based on at least some of the characteristics of the graphic object.

[0151] In operation 1213, in one embodiment, the electronic device (201) can modify the second preview image.

[0152] In operation 1215, in one embodiment, the electronic device (201) may display a second preview image modified in the second part.

[0153] Referring to FIG. 13c, an example is shown in which an e-type electronic device (1311) displays a graphic object. Referring to reference numeral 1313, the e-type electronic device (1311) can display (1351) a graphic object (381) on the first preview image (333) based on user input regarding the first preview image (333). The e-type electronic device (1311) can modify the second preview image (343) based on at least some of the characteristics of the generated graphic object (381). For example, the e-type electronic device (1311) can change the brightness for each region of the second preview image (343) (or corresponding to at least some of the pixels of the second preview image (343)). Referring to reference numeral 1341, the e-type electronic device (1311) can display a preview image (1343a) obtained through a first image sensor (e.g., first image sensor (211)) and a preview image (1343b) obtained through a second image sensor (e.g., second image sensor (213)) on the right area of ​​the main display (1302) when partially unfolded. Because the e-type electronic device (1311) is partially unfolded, the field of view of the first image sensor may overlap at least partially with the field of view of the second image sensor. The e-type electronic device (1311) can display a graphic object (381) on the preview image (1343b) obtained through the second image sensor based on a graphic object (381) created on the preview image (1343a) obtained through the first image sensor.

[0154] Referring to FIG. 13d, an example is shown in which an e-type electronic device (1311) displays a graphic object. Referring to reference numeral 1361, the e-type electronic device (1311) is partially unfolded so that the field of view of the first image sensor (e.g., the first image sensor (211)) may overlap at least partially with the field of view of the second image sensor (e.g., the second image sensor (213)). The e-type electronic device (1311) can display a graphic object (381) on a preview image (1343b) obtained through the second image sensor based on a graphic object (381) generated on a preview image (1343a) obtained through the first image sensor. Referring to reference numeral 1335, the e-type electronic device (1311) can display a preview image (1345) containing a graphic object (381) and a preview image (1363) obtained through a third image sensor (1360) on the entire area of ​​the main display (1302) when fully unfolded. The preview image (1345) displayed on the entire area of ​​the main display (1302) may be a preview image obtained through a first image sensor. The e-type electronic device (1311) can modify (or regenerate) the preview image (1363) obtained through the third image sensor (1360) based on the characteristics of the graphic object (381). Referring to reference numeral 1365, the e-type electronic device (1311) can display a preview image (1367) containing a graphic object (381) and a preview image (1363) obtained through a third image sensor (1360) on the left area of ​​the main display (1302) when partially folded. The preview image (1367) displayed on at least a portion of the main display (1302) may be, for example, a preview image obtained through a second image sensor (e.g., a second image sensor (213)).The preview image (1363) obtained through the third image sensor (1360) may be modified (or regenerated) based on at least some of the characteristics of the graphic object (381) included in the preview image (1367). In the examples of FIGS. 13a through 13d, cases where there are two or three image sensors are illustrated, but an electronic device including three or more cameras can improve the user experience by applying the effect applied to the preview image to the remaining preview images as well.

[0155] Referring to FIG. 13e, a real-time change in the preview image is shown while the e-type electronic device (1311) is partially unfolded. Referring to reference numerals 1313 and 1371, the preview image obtained through the second image sensor (213) may change while the e-type electronic device (1311) transitions from a fully folded state to a partially unfolded state. For example, referring to reference numeral 1313, the e-type electronic device (1311) may display a preview image (333) obtained through the first image sensor (e.g., the first image sensor (211)) and a preview image (343) obtained through the second image sensor (213) on the sub-display (1301). The preview image (333) obtained through the first image sensor may be, for example, a preview image in which the background (350) is captured. The preview image (343) obtained through the second image sensor may be a preview image in which subjects (360), including a user, are captured. The e-type electronic device (1311) may display a first graphic object (381) on a preview image (333) obtained through the first image sensor based on user input for creating a graphic object. In reference numeral 1371, the e-type electronic device (1311) may display a preview image (1373b) in which a side subject (1370) is captured based on the first hinge (e.g., the first hinge (1303a)) being partially unfolded. The e-type electronic device (1311) can display a preview image (1373a) obtained through the first image sensor (211) while the subject of the preview image obtained through the second image sensor (213) changes from subjects (360) including the user to a side subject (1370). An effect based on the characteristics of the first graphic object (381) on the preview image (1373a) can be applied to the preview image (1373b) in which the side subject (1370) is captured.For example, the e-type electronic device (1311) can change the brightness of the pixels of a preview image (1373b) in which a side subject (1370) is captured, based on the characteristics of the first graphic object (381) corresponding to the sun. In reference numeral 1361, the e-type electronic device (1311) can display a preview image (1343a) obtained through a first image sensor and a preview image (1343b) obtained through a second image sensor on at least a portion of the main display (1302). Due to the e-type electronic device (1311) being partially unfolded (e.g., the first hinge (1303a) being fully unfolded), the field of view of the first image sensor (e.g., the first image sensor (211)) may overlap at least partially with the field of view of the second image sensor (e.g., the second image sensor (213)). The e-type electronic device (1311) can display the first graphic object (381) on the preview image (1343b) obtained through the second image sensor based on the characteristics of the first graphic object (381) generated on the preview image (1343a) obtained through the first image sensor.

[0156] FIG. 14 is a drawing illustrating a method for an electronic device to provide a preview image according to one embodiment.

[0157] In one embodiment, object effects may be applied to preview images captured by rear cameras (1401, 1403). Referring to reference numeral 1410, the first rear camera (1401) and the second rear camera (1403) may capture subjects (350) in front. The field of view (1411) of the first rear camera (1401) may be wider than the field of view (1413) of the second rear camera (1403). Referring to reference numeral 1420, the electronic device (201) may display a first preview image (1421) on a first part (1431) and a second preview image (1423) on a second part (1433). The electronic device (201) may overlay the second preview image (1423) on the first preview image (1421). The electronic device (201) can regenerate the first preview image (1421) based on user input for creating a graphic object for the first preview image (1421). The electronic device (201) can regenerate the second preview image (1423) based on the regenerated first preview image (1421) (or at least some of the characteristics of the graphic object).

[0158] FIG. 15 is a diagram illustrating a generative artificial intelligence model according to one embodiment.

[0159] Referring to FIG. 15, the user query / response interface (1510), application / service component (1530), knowledge repository (1520), AI framework (1540), and generative AI model (1560) (e.g., AI model (400) of FIG. 4) may be stored in memory (230) (e.g., memory (230) of FIG. 2) or on a separate server. At least some of the user query / response interface (1510), application / service component (1530), knowledge repository (1520), AI framework (1540), or generative AI model (1560) may be implemented in software or in hardware.

[0160] According to one embodiment, a user query / response interface (1510) may receive user input. The user input may be in the form of natural language, images, and / or videos, but is not limited thereto. Additionally, context information may be transmitted along with the user input. The context information may include various additional information at the time of user input. For example, the additional information may include information about the application currently being used by the user or the user's location information. Furthermore, the user input may be in a mixed form of the aforementioned natural language, images, sounds, and context information. Additionally, the user input may be in a non-natural language form, such as selecting a menu. The user query / response interface (1510) may output results from a generative artificial intelligence system to the user. The output may be in the form of natural language or specific content, and may also be provided in the form of an action requested by the user. The user query / response interface (1510) may output results from a generative artificial intelligence system to the user. The output can be in the form of natural language or specific content, and it can also be provided in a form such as the action requested by the user.

[0161] The AI ​​framework (1540) can receive input from the user and coordinate and control each component necessary to perform the user's intent based on the user's query.

[0162] User input received from the user query / response interface (1510) can be transmitted to a prompt design component (1541). The prompt design component (1541) can be used to generate prompts suitable for inputting user input into a large language model (LLM), a large vision model (LVM), or a large multimodal model (LMM). The prompt design component (1541) may be an AI component that uses machine learning algorithms or neural networks to develop better prompts over time. Based on user input, the prompt design component (1541) can generate prompts by accessing a knowledge component containing user preference data, a prompt library, and prompt examples, and can transmit the generated prompts to the large language model (LLM) or large multimodal model (LMM).

[0163] The API / Plug-in management component (1542) can perform the role of communicating with external information when there is a request for additional information when user input is passed as input to a generative model. The API / Plug-in management component (1542) establishes a channel to communicate with the outside of the AI ​​Interface via the API, and through the established channel, it can enable access to various data sources (e.g., knowledge repository (1520)). Additionally, if the API / Plug-in management component (1542) needs to perform an action that executes the user input as a final step rather than an intermediate result in an application or service, it can request that action from the application / service component (1530) via the API. The information obtained from the outside may be used to generate a prompt in the prompt design component (1541) along with the user input, or it may be passed as input to the generative model.

[0164] The output modification component (or refiner component) (1543) can fine-tune the output of the generative model. For example, the output modification component (1543) can verify whether the content generated through the LLM and / or LMM is irrelevant, contains biased content, or contains harmful content. Additionally, the output modification component (1543) can determine the extent to which the output matches the desired result and, if necessary, proceed with the additional process. Furthermore, the output modification component (1543) can configure and provide hints to the user to avoid unwanted output.

[0165] A generative AI model (1560) generally refers to an artificial intelligence neural network that generates new forms of data based on user input information. A generative AI model (1560) may include models that generate images and / or models that generate language. Models that generate images include, but are not limited to, GANs (generative adversarial networks) and VAEs (variational autoencoders), and examples include Diffusion-based generative models that use VAEs and Transformer structures. Models that generate language are models trained to output the most statistically appropriate output value based on input values, and examples include models such as CHAT-GPT 3 and CHAT-GPT 4. There are also LMMs (large multimodal models) that can recognize various forms of data input, such as text, images, and audio, and generate new data corresponding to them.

[0166] In one embodiment, an electronic device (e.g., electronic device (201)) providing a multi-preview function can solve the problem that an effect applied to one preview image cannot be applied to another preview image. The electronic device can, for example, apply an effect based on the characteristics of a graphic object to a second preview image based on the characteristics of a graphic object created on a first preview image.

[0167] The technical problems to be solved in this disclosure are not limited to those mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art to which this disclosure pertains.

[0168] An electronic device (e.g., electronic device (201)) according to one embodiment may include a first image sensor (e.g., first image sensor (211)), a second image sensor (e.g., second image sensor (213)), a touchscreen display (e.g., touchscreen display (220)), at least one processor (e.g., processor (240)), and a memory (e.g., memory (230)) for storing instructions. When the instructions are executed individually or collectively by the at least one processor (240), the electronic device (201) may cause the electronic device (201) to acquire a first preview image through the first image sensor (211). When the instructions are executed individually or collectively by the at least one processor (240), the electronic device (201) may cause the electronic device (201) to acquire a second preview image through the second image sensor (213). When the above instructions are executed individually or collectively by the at least one processor (240), they may cause the electronic device (201) to display the first preview image on a first part of the touchscreen display (220) and the second preview image on a second part of the touchscreen display (220) through the touchscreen display (220). When the above instructions are executed individually or collectively by the at least one processor (240), they may cause the electronic device (201) to receive user input for creating an AI object for the first preview image. When the above instructions are executed individually or collectively by the at least one processor (240), they may cause the electronic device (201) to create the AI ​​object using a machine learning model based on the user input.The machine learning model may include a generative AI model trained to output at least one AI object based on text, voice, or drawing input. When the instructions are executed individually or collectively by the at least one processor (240), the electronic device (201) may cause the first preview image and the second preview image to be modified using the machine learning model based on the characteristics of the generated AI object. When the instructions are executed individually or collectively by the at least one processor (240), the electronic device (201) may cause the modified first preview image to be displayed in the first part and the modified second preview image to be displayed in the second part.

[0169] In one embodiment, when the instructions are executed individually or collectively by the at least one processor (240), the electronic device (201) may cause to receive other user input regarding the modified second preview image. When the instructions are executed individually or collectively by the at least one processor (240), the electronic device (201) may cause to generate another AI object at least partially based on the other user input. When the instructions are executed individually or collectively by the at least one processor (240), the electronic device (201) may cause to modify the modified second preview image using the machine learning model at least partially based on another characteristic of the other AI object. When the above instructions are executed individually or collectively by the at least one processor (240), the electronic device (201) may cause the AI ​​object and the first preview image modified based on the other AI object to display based on the other AI object.

[0170] In one embodiment, when the instructions are executed individually or collectively by the at least one processor (240), the electronic device (201) may be caused to perform the operation of displaying the first preview image and the second preview image by overlaying the second preview image on the first preview image.

[0171] In one embodiment, the user input may include a drawing input. When the instructions are executed individually or collectively by the at least one processor (240), the electronic device (201) may cause at least one object corresponding to the drawing input to be displayed on the first preview image via the touchscreen display (220). When the instructions are executed individually or collectively by the at least one processor (240), the electronic device (201) may cause the AI ​​object to be generated via machine learning, at least based on a part of the at least one object.

[0172] In one embodiment, the first image sensor (211) may be placed on a first surface of the electronic device (201). The second image sensor (213) and the touchscreen display (220) may be placed on a second surface opposite to the first surface. When the instructions are executed individually or collectively by the at least one processor (240), they may cause the electronic device (201) to modify a second preview image at least based on the location where the second image sensor (213) is placed on the electronic device (201).

[0173] In one embodiment, when the instructions are executed individually or collectively by the at least one processor (240), the electronic device (201) may be caused to receive other user input for capturing at least a portion of the modified first preview image and the modified second preview image. When the instructions are executed individually or collectively by the at least one processor (240), the electronic device (201) may be caused to save a captured image including at least a portion of the modified first preview image and the modified second preview image based on the other user input. When the instructions are executed individually or collectively by the at least one processor (240), the electronic device (201) may be caused to save metadata associated with the AI ​​object together with the captured image.

[0174] In one embodiment, when the instructions are executed individually or collectively by the at least one processor (240), the electronic device (201) may cause the modified first preview image to be displayed in the first portion. When the instructions are executed individually or collectively by the at least one processor (240), the electronic device (201) may cause the modified second preview image to be displayed in the second portion after the modified first preview image has been displayed.

[0175] In one embodiment, when the instructions are executed individually or collectively by the at least one processor (240), the electronic device (201) may cause another user input to be received for displaying the modified second preview image. When the instructions are executed individually or collectively by the at least one processor (240), the electronic device (201) may cause the electronic device (201) to perform an operation to display the modified second preview image in response to the other user input.

[0176] In one embodiment, the position where the AI ​​object is displayed on the first preview image may be changed based on the movement of the electronic device. When the instructions are executed individually or collectively by the at least one processor (240), the electronic device (201) may be caused to refrain from displaying the modified second preview image in the second part when the AI ​​object moves out of the first preview image based on the movement of the electronic device (201). When the instructions are executed individually or collectively by the at least one processor (240), the electronic device (201) may be caused to display the second preview image in the second part through the touchscreen display (220).

[0177] A method according to one embodiment may include an operation of acquiring a first preview image through a first image sensor (211) of an electronic device (201). The method may include an operation of acquiring a second preview image through a second image sensor (213) of the electronic device (201). The method may include an operation of displaying the first preview image on a first part of the touchscreen display (220) and displaying the second preview image on a second part of the touchscreen display (220) through the touchscreen display (220) of the electronic device (201). The method may include an operation of receiving user input for generating an AI object with respect to the first preview image. The method may include an operation of generating the AI ​​object using a machine learning model based on the user input. The machine learning model may include a generative AI model trained to output at least one AI object based on text, voice, or drawing input. The above method may include an operation of modifying the first preview image and the second preview image using the machine learning model based on the characteristics of the generated AI object. The above method may include an operation of displaying the modified first preview image in the first part and displaying the modified second preview image in the second part.

[0178] In one embodiment, the method may further include an action of receiving another user input regarding the modified second preview image. The method may further include an action of generating another AI object based at least partially on the other user input. The method may further include an action of modifying the modified second preview image using the machine learning model based at least partially on another characteristic of the other AI object. The method may further include an action of modifying the modified first preview image using the machine learning model based at least partially on the other AI object.

[0179] In one embodiment, the operation of displaying the first preview image on a first part of the touchscreen display (220) and displaying the second preview image on a second part of the touchscreen display (220) through the touchscreen display (220) of the electronic device (201) may include the operation of displaying the first preview image and the second preview image by overlaying the second preview image on the first preview image.

[0180] In one embodiment, the user input may include a drawing input. The operation of generating the AI ​​object using a machine learning model based on the user input may include the operation of displaying at least one object corresponding to the drawing input on the first preview image through the touchscreen display (220). The operation of generating the AI ​​object using a machine learning model based on the user input may include the operation of generating the AI ​​object through the machine learning model, at least based on a part of the at least one object.

[0181] In one embodiment, the first image sensor (211) may be placed on a first surface of the electronic device (201). The second image sensor (213) and the touchscreen display (220) may be placed on a second surface opposite to the first surface. The operation of modifying the first preview image and the second preview image using the machine learning model based on the characteristics of the generated AI object may include the operation of modifying the second preview image based at least on the position where the second image sensor (213) is placed on the electronic device (201).

[0182] In one embodiment, the method may further include an operation of receiving another user input for capturing at least a portion of the modified first preview image and the modified second preview image. The method may further include an operation of storing a captured image comprising at least a portion of the modified first preview image and the modified second preview image based on the other user input. The method may further include an operation of storing metadata associated with the AI ​​object together with the captured image.

[0183] In one embodiment, the operation of displaying the modified first preview image in the first part and the modified second preview image in the second part may include the operation of displaying the modified first preview image in the first part. The operation of displaying the modified first preview image in the first part and the modified second preview image in the second part may include the operation of displaying the modified second preview image in the second part after displaying the modified first preview image.

[0184] According to one embodiment, a recording medium recording computer-executable instructions may be provided. When executed by a processor (240) of an electronic device (201), the computer-executable instructions may cause the electronic device (201) to acquire a first preview image through a first image sensor (211) of the electronic device (201). When executed by a processor (240) of an electronic device (201), the computer-executable instructions may cause the electronic device (201) to acquire a second preview image through a second image sensor (213) of the electronic device (201). When the above computer-executable instructions are executed by the processor (240) of the electronic device (201), the electronic device (201) may cause the electronic device (201) to display the first preview image on a first part of the touchscreen display (220) and the second preview image on a second part of the touchscreen display (220) through the touchscreen display (220) of the electronic device (201). When the above computer-executable instructions are executed by the processor (240) of the electronic device (201), the electronic device (201) may cause the electronic device (201) to receive user input for creating an AI object with respect to the first preview image. When the above computer-executable instructions are executed by the processor (240) of the electronic device (201), the electronic device (201) may cause the electronic device (201) to create the AI ​​object using a machine learning model based on the user input. The machine learning model may include a generative AI model trained to output at least one AI object based on text, voice, or drawing input.When the above computer-executable instructions are executed by the processor (240) of the electronic device (201), the electronic device (201) may cause the first preview image and the second preview image to be modified using the machine learning model based on the characteristics of the generated AI object. When the above computer-executable instructions are executed by the processor (240) of the electronic device (201), the electronic device (201) may cause the modified first preview image to be displayed in the first part and the modified second preview image to be displayed in the second part.

[0185] An electronic device (201) according to one embodiment may include a first image sensor (211), a second image sensor (213), a touchscreen display (220), at least one processor (240), and a memory (230) for storing instructions. When the instructions are executed individually or collectively by the at least one processor (240), the electronic device (201) may cause the electronic device (201) to acquire a first preview image through the first image sensor (211). When the instructions are executed individually or collectively by the at least one processor (240), the electronic device (201) may cause the electronic device (201) to acquire a second preview image through the second image sensor (213). When the above instructions are executed individually or collectively by the at least one processor (240), the electronic device (201) may cause the first preview image to be displayed on a first part of the touchscreen display (220) and the second preview image to be displayed on a second part of the touchscreen display (220) through the touchscreen display (220). When the above instructions are executed individually or collectively by the at least one processor (240), the electronic device (201) may cause the first preview image to be input into a generative AI model trained to display a graphic object on the input preview image based on receiving user input for displaying a first object on the first preview image, thereby causing the first preview image to be obtained based on information output from the generative AI model.When the above instructions are executed individually or collectively by the at least one processor (240), the electronic device (201) may be caused to obtain a modified second preview image based on information output from the generative AI model by inputting information associated with the characteristics of the first object and the second preview image into the generative AI. When the above instructions are executed individually or collectively by the at least one processor (240), the electronic device (201) may be caused to display the modified first preview image in the first part and display the modified second preview image in the second part through the touchscreen display (220).

[0186] An electronic device (201) according to one embodiment may include a first image sensor (211), a second image sensor (213), a touchscreen display (220), at least one processor (240), and a memory (230) for storing instructions. When the instructions are executed individually or collectively by the at least one processor (240), the electronic device (201) may cause the electronic device (201) to acquire a first preview image through the first image sensor (211). When the instructions are executed individually or collectively by the at least one processor (240), the electronic device (201) may cause the electronic device (201) to acquire a second preview image through the second image sensor (213). When the above instructions are executed individually or collectively by the at least one processor (240), they may cause the electronic device (201) to display the first preview image on a first part of the touchscreen display (220) and the second preview image on a second part of the touchscreen display (220) through the touchscreen display (220). When the above instructions are executed individually or collectively by the at least one processor (240), they may cause the electronic device (201) to receive user input to create at least one graphic object on the first preview image through the touchscreen display (220). When the above instructions are executed individually or collectively by the at least one processor (240), they may cause the electronic device (201) to create a graphic object on the first preview image based on at least a portion of the user input.When the above instructions are executed individually or collectively by the at least one processor (240), they may cause the electronic device (201) to display a modified first preview image containing the graphic object on a first part of the touchscreen display (220) through the touchscreen display (220) based on at least some of the characteristics of the graphic object. When the above instructions are executed individually or collectively by the at least one processor (240), they may cause the electronic device (201) to modify a second preview image based on at least some of the characteristics of the graphic object. When the above instructions are executed individually or collectively by the at least one processor (240), they may cause the electronic device (201) to display the modified second preview image on the second part through the touchscreen display (220).

[0187] In one embodiment, the electronic device (201) may be a multi-foldable electronic device. The first image sensor (211) may be placed on a first housing on a first side of the foldable electronic device. The second image sensor (213) may be placed on a second housing different from the first housing on the first side of the foldable electronic device. When the instructions are executed individually or collectively by the at least one processor (240), the electronic device (201) may be caused to acquire a first preview image in which a subject in a first direction is captured through the first image sensor (211) and to acquire a first preview image in which a subject in a second direction opposite to the first direction is captured through the second image sensor (213) while the electronic device (201) is partially folded. When the above instructions are executed individually or collectively by the at least one processor (240), they may cause the electronic device (201) to acquire a first preview image and a second preview image of the subject in the first direction through the first image sensor (201) and the second image sensor (213) in the unfolded state of the electronic device (201). When the above instructions are executed individually or collectively by the at least one processor (240), they may cause the electronic device (201) to acquire a first preview image of the subject in the second direction through the first image sensor (211) and acquire a first preview image of the subject in the first direction through the second image sensor (213) in the fully folded state of the electronic device (201).

[0188] In one embodiment, the electronic device (201) can provide a vivid and natural multi-preview image by going beyond simply displaying a graphic object (e.g., graphic object (381)) on a first preview image (333) and applying an effect associated with the graphic object to a second preview image (343) based on the characteristics of the graphic object.

[0189] The effects obtainable from the present disclosure are not limited to those mentioned above, and other unmentioned effects will be clearly understood by those skilled in the art to which the present disclosure belongs.

[0190] Various embodiments of the present document may be implemented as software (e.g., program (140)) comprising one or more instructions stored in a storage medium (e.g., internal memory (136) or external memory (138)) readable by a machine (e.g., electronic device (101)). For example, a processor (e.g., processor (120)) of the machine (e.g., electronic device (101)) may call at least one of the one or more instructions stored in the storage medium and execute it. This enables the machine to be operated to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code that can be executed by an interpreter. The storage medium readable by the machine may be provided in the form of a non-transitory storage medium. Here, 'non-temporary' simply means that the storage medium is a tangible device and does not contain a signal (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily.

[0191] According to one embodiment, the method according to the various embodiments disclosed herein may be provided by being included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or an application store (e.g., Play Store). TM It can be distributed online (e.g., downloaded or uploaded) through ) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created on a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0192] According to various embodiments, each component (e.g., module or program) of the components described above may include a singular or multiple entities, and some of the multiple entities may be separated and placed in other components. According to various embodiments, one or more of the components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Generally or additionally, multiple components (e.g., module or program) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each of the multiple components in the same or similar manner as those performed by the corresponding component among the multiple components prior to integration. According to various embodiments, operations performed by the module, program, or other components may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

Claims

1. In an electronic device (201), First image sensor (211); Second image sensor (213); Touchscreen display (220); At least one processor (240); and The electronic device (201) includes a memory (230) for storing instructions, and when the instructions are executed individually or collectively by the at least one processor (240): A first preview image is obtained through the first image sensor (211), and A second preview image is obtained through the second image sensor (213), and The first preview image is displayed on a first part of the touchscreen display (220) through the touchscreen display (220), and the second preview image is displayed on a second part of the touchscreen display (220). Receiving user input to create an AI object for the first preview image above, and Generating the AI ​​object using a machine learning model based on the above user input—the machine learning model includes a generative AI model trained to output at least one AI object based on text, voice, or drawing input—, Based on the characteristics of the AI ​​object generated above, the first preview image and the second preview image are modified using the machine learning model, and An electronic device (201) that causes the modified first preview image to be displayed in the first part and the modified second preview image to be displayed in the second part.

2. In Paragraph 1, When the above instructions are executed individually or collectively by the at least one processor (240), the electronic device (201) causes, Receiving other user input regarding the above modified second preview image, and Based at least partially on the above other user input, create other AI objects, and Based at least partially on another characteristic of the other AI object mentioned above, the machine learning model is used to modify the modified second preview image, and An electronic device (201) that causes the modified first preview image to be modified using the machine learning model, based at least partially on the other AI object mentioned above.

3. In Paragraph 1 or 2, The above instructions, when executed individually or collectively by the at least one processor (240), cause the electronic device (201) to perform the operation of displaying the first preview image and the second preview image by overlaying the second preview image on the first preview image.

4. In any one of paragraphs 1 to 3, The above user input includes drawing input, and When the above instructions are executed individually or collectively by the at least one processor (240), the electronic device (201) causes, Through the touchscreen display (220), at least one object corresponding to the drawing input is displayed on the first preview image, and An electronic device (201) that causes the AI ​​object to be generated based on at least a part of at least one object through the machine learning model.

5. In any one of paragraphs 1 to 4, The first image sensor (211) is disposed on a first surface of the electronic device (201), and the second image sensor (213) and the touchscreen display (220) are disposed on a second surface opposite to the first surface. The above instructions, when executed individually or collectively by the at least one processor (240), cause the electronic device (201) to modify a second preview image at least based on the location where the second image sensor (213) is placed in the electronic device (201).

6. In any one of paragraphs 1 to 5, When the above instructions are executed individually or collectively by the at least one processor (240), the electronic device (201) causes, Receiving other user input for capturing at least a portion of the modified first preview image and the modified second preview image, and Based on the other user input above, a captured image including at least a portion of the modified first preview image and the modified second preview image is stored, and An electronic device (201) that causes metadata associated with the above AI object to be stored together with the above captured image.

7. In any one of paragraphs 1 through 6, When the above instructions are executed individually or collectively by the at least one processor (240), the electronic device (201) causes, Display the above modified first preview image in the above first part, and An electronic device (201) that causes the modified second preview image to be displayed in the second part after the modified first preview image is displayed.

8. In any one of paragraphs 1 through 7, When the above instructions are executed individually or collectively by the at least one processor (240), the electronic device (201) receives another user input for displaying the modified second preview image, and An electronic device (201) that causes to perform the operation of displaying the modified second preview image in response to the above another user input.

9. In any one of paragraphs 1 through 8, The position where the AI ​​object is displayed on the first preview image is changed based on the movement of the electronic device, and When the above instructions are executed individually or collectively by the at least one processor (240), the electronic device (201) refrains from displaying the modified second preview image in the second part when the AI ​​object moves out of the first preview image based on the movement of the electronic device (201). An electronic device (201) that causes the second preview image to be displayed in the second part through the touchscreen display (220).

10. Regarding the method, The operation of acquiring a first preview image through the first image sensor (211) of the electronic device (201); The operation of acquiring a second preview image through the second image sensor (213) of the electronic device (201); The operation of displaying the first preview image on a first part of the touchscreen display (220) and displaying the second preview image on a second part of the touchscreen display (220) through the touchscreen display (220) of the electronic device (201); The operation of receiving user input to create an AI object for the first preview image above; The operation of generating the AI ​​object using a machine learning model based on the above user input—the machine learning model includes a generative AI model trained to output at least one AI object based on text, voice, or drawing input; An operation to modify the first preview image and the second preview image using the machine learning model based on the characteristics of the AI ​​object generated above; and The operation of displaying the modified first preview image in the first part and displaying the modified second preview image in the second part. A method including 11. In Paragraph 10, The operation of receiving other user input regarding the above-mentioned modified second preview image; An action of generating another AI object based at least partially on the other user input mentioned above; An operation to modify the modified second preview image using the machine learning model, based at least partially on another characteristic of the other AI object; and A method further comprising the operation of modifying the modified first preview image using the machine learning model, based at least partially on the other AI object.

12. In Article 10 or Article 11, A method for displaying a first preview image on a first part of a touchscreen display (220) and displaying a second preview image on a second part of a touchscreen display (220) through a touchscreen display (220) of the electronic device (201), wherein the operation of displaying the first preview image and the second preview image by overlaying the second preview image on the first preview image.

13. In any one of paragraphs 10 through 12, The above user input includes drawing input, and The operation of generating the AI ​​object using a machine learning model based on the above user input is, The operation of displaying at least one object corresponding to the drawing input on the first preview image through the touchscreen display (220); and A method comprising the operation of generating the AI ​​object based at least on a part of the at least one object through the machine learning model.

14. In any one of paragraphs 10 through 13, The first image sensor (211) is disposed on a first surface of the electronic device (201), and the second image sensor (213) and the touchscreen display (220) are disposed on a second surface opposite to the first surface. A method in which the operation of modifying the first preview image and the second preview image using the machine learning model based on the characteristics of the generated AI object includes at least the operation of modifying the second preview image based on the position where the second image sensor (213) is placed in the electronic device (201).

15. In a recording medium recording computer-executable instructions, the computer-executable instructions, when executed by a processor (240) of an electronic device (201), cause the electronic device (201): A first preview image is obtained through the first image sensor (211) of the electronic device (201), and A second preview image is obtained through the second image sensor (213) of the electronic device (201), and The first preview image is displayed on a first part of the touchscreen display (220) through the touchscreen display (220) of the electronic device (201), and the second preview image is displayed on a second part of the touchscreen display (220). Receiving user input to create an AI object for the first preview image above, and Generating the AI ​​object using a machine learning model based on the above user input—the machine learning model includes a generative AI model trained to output at least one AI object based on text, voice, or drawing input—, Based on the characteristics of the AI ​​object generated above, the first preview image and the second preview image are modified using the machine learning model, and A recording medium that causes the modified first preview image to be displayed in the first part and the modified second preview image to be displayed in the second part.

Images