Electronic device and operation method for generating UI object

Abstract

Disclosed are an electronic device and an operation method for generating a UI object. The electronic device according to an embodiment of the present disclosure may comprise a memory for storing instructions. The electronic device may comprise a processor for executing the instructions. The instructions, when executed by the processor, may cause the electronic device to acquire an image. The instructions, when executed by the processor, may cause the electronic device to obtain first feature information of an app for which a target UI object is to be generated. The instructions, when executed by the processor, may cause the electronic device to extract second feature information from the image. The instructions, when executed by the processor, may cause the electronic device to generate the target UI object related to the app on the basis of the first feature information and the second feature information.

Description

Electronic device and method for creating UI objects

[0001] An electronic device and method of operating a UI object are disclosed.

[0002] As times change, more and more people are seeking to express their individuality. Consequently, electronic devices like smartphones, which have become essential in everyday life, are becoming one of the various ways to express individuality. For example, people are using images that best express their individuality as the wallpaper on their electronic devices like smartphones. However, some people are going beyond this and are customizing UI objects, such as app icons, provided by default by app providers.

[0003] The background technology described above is possessed or acquired during the process of deriving the present disclosure, and cannot necessarily be said to be a publicly known technology disclosed to the general public prior to the filing of the present disclosure.

[0004] An electronic device according to an embodiment of the present disclosure may include a memory that stores instructions. The electronic device may include a processor that executes the instructions. The instructions, when executed by the processor, may cause the electronic device to obtain an image. The instructions, when executed by the processor, may cause the electronic device to obtain first characteristic information of an app for generating a target UI object. The instructions, when executed by the processor, may cause the electronic device to extract second characteristic information from the image. The instructions, when executed by the processor, may cause the electronic device to generate the target UI object associated with the app based on the first characteristic information and the second characteristic information.

[0005] An electronic device according to one embodiment of the present disclosure may include a memory that stores instructions. The electronic device may include a processor that executes the instructions. The instructions, when executed by the processor, may cause the electronic device to obtain at least one first image. The instructions, when executed by the processor, may cause the electronic device to generate a second image using at least some of information and auxiliary information related to an app that will generate at least one target UI object based on at least some of features of the at least one first image. The instructions, when executed by the processor, may cause the electronic device to generate at least one target UI object associated with performing a function of the app based on the second image.

[0006] A method of operating an electronic device according to one embodiment of the present disclosure may include an operation of acquiring an image. The method of operating the electronic device may include an operation of acquiring first characteristic information of an app for generating a target UI object. The method of operating the electronic device may include an operation of extracting second characteristic information from the image. The method of operating the electronic device may include an operation of generating the target UI object related to the app based on the first characteristic information and the second characteristic information.

[0007] A computer-readable recording medium according to one embodiment of the present disclosure can store one or more programs including commands for performing the operating method of the electronic device described above.

[0008] A method of operating an electronic device according to one embodiment of the present disclosure may include an operation of acquiring at least one first image. The method of operating the electronic device may include an operation of generating a second image using at least a portion of information and auxiliary information related to an app that will generate at least one target UI object based on at least a portion of features of the at least one first image. The electronic device may include an operation of generating the at least one target UI object associated with the performance of a function of the app based on the second image.

[0009] A computer-readable recording medium according to one embodiment of the present disclosure can store one or more programs including commands for performing the operating method of the electronic device described above.

[0010] FIG. 1 is a block diagram of an electronic device within a network environment according to various embodiments.

[0011] FIG. 2 is a diagram for explaining the creation of a target UI object according to one embodiment of the present disclosure.

[0012] FIG. 3 and FIG. 4 are drawings for explaining an example of creating a target UI object according to one embodiment of the present disclosure.

[0013] FIG. 5 is a diagram for explaining the creation of a target UI object when multiple users use an electronic device according to one embodiment of the present disclosure.

[0014] FIGS. 6 to 9 are drawings for explaining a generated target UI object according to one embodiment of the present disclosure.

[0015] FIG. 10 is a diagram illustrating sharing of a generated target UI object according to one embodiment of the present disclosure.

[0016] FIG. 11 is a diagram illustrating target UI objects with different degrees of personalization according to one embodiment of the present disclosure.

[0017] FIG. 12 is a drawing for explaining a 3D target UI object according to one embodiment of the present disclosure.

[0018] FIG. 13 is a diagram for explaining a case where a UI object according to one embodiment of the present disclosure is a widget.

[0019] FIG. 14 is a flowchart for explaining an operation method of an electronic device according to one embodiment of the present disclosure.

[0020] FIG. 15 is a block diagram of a generative artificial intelligence system according to various embodiments.

[0021] Hereinafter, embodiments will be described in detail with reference to the attached drawings. In the description with reference to the attached drawings, identical components are assigned the same reference numerals regardless of the drawing numbers, and redundant descriptions thereof will be omitted.

[0022] FIG. 1 is a block diagram of an electronic device (101) within a network environment (100) according to various embodiments. Referring to FIG. 1, in the network environment (100), the electronic device (101) may communicate with the electronic device (102) via a first network (198) (e.g., a short-range wireless communication network), or may communicate with at least one of the electronic device (104) or the server (108) via a second network (199) (e.g., a long-range wireless communication network). In one embodiment, the electronic device (101) may communicate with the electronic device (104) via the server (108). According to one embodiment, the electronic device (101) may include a processor (120), a memory (130), an input module (150), an audio output module (155), a display module (160), an audio module (170), a sensor module (176), an interface (177), a connection terminal (178), a haptic module (179), a camera module (180), a power management module (188), a battery (189), a communication module (190), a subscriber identification module (196), or an antenna module (197). In some embodiments, the electronic device (101) may omit at least one of these components (e.g., the connection terminal (178)), or may have one or more other components added. In some embodiments, some of these components (e.g., the sensor module (176), the camera module (180), or the antenna module (197)) may be integrated into one component (e.g., the display module (160)).

[0023] The processor (120) may, for example, execute software (e.g., a program (140)) to control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) and perform various data processing or operations. According to one embodiment, as at least a part of the data processing or operations, the processor (120) may store commands or data received from other components (e.g., a sensor module (176) or a communication module (190)) in a volatile memory (132), process the commands or data stored in the volatile memory (132), and store result data in a non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or an auxiliary processor (123) (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor) that can operate independently or together with the main processor (121). For example, when the electronic device (101) includes the main processor (121) and the auxiliary processor (123), the auxiliary processor (123) may be configured to use less power than the main processor (121) or to be specialized for a given function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as a part thereof.

[0024] The auxiliary processor (123) may control at least a portion of functions or states associated with at least one component (e.g., a display module (160), a sensor module (176), or a communication module (190)) of the electronic device (101), for example, on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (123) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (180) or a communication module (190)). In one embodiment, the auxiliary processor (123) (e.g., a neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. This learning can be performed, for example, in the electronic device (101) itself where the artificial intelligence model is executed, or can be performed through a separate server (e.g., server (108)). The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include a plurality of artificial neural network layers.The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to, or alternatively to, a hardware structure, an artificial intelligence model may include a software structure.

[0025] The memory (130) can store various data used by at least one component (e.g., processor (120) or sensor module (176)) of the electronic device (101). The data can include, for example, software (e.g., program (140)) and input data or output data for commands related thereto. The memory (130) can include volatile memory (132) or non-volatile memory (134).

[0026] The program (140) may be stored as software in the memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0027] The input module (150) can receive commands or data to be used in a component of the electronic device (101) (e.g., a processor (120)) from an external source (e.g., a user) of the electronic device (101). The input module (150) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0028] The audio output module (155) can output audio signals to the outside of the electronic device (101). The audio output module (155) can include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. In one embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

[0029] The display module (160) can visually provide information to an external party (e.g., a user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by the touch.

[0030] The audio module (170) can convert sound into an electrical signal, or vice versa, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150), output sound through the sound output module (155), or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphone) directly or wirelessly connected to the electronic device (101).

[0031] The sensor module (176) can detect the operating status (e.g., power or temperature) of the electronic device (101) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. According to one embodiment, the sensor module (176) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0032] The interface (177) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (101) with an external electronic device (e.g., the electronic device (102)). In one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0033] The connection terminal (178) may include a connector through which the electronic device (101) may be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0034] The haptic module (179) can convert electrical signals into mechanical stimuli (e.g., vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic sensations. According to one embodiment, the haptic module (179) can include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

[0035] The camera module (180) can capture still images and videos. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0036] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

[0037] A battery (189) may power at least one component of the electronic device (101). In one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0038] The communication module (190) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may operate independently from the processor (120) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (194) (e.g., a local area network (LAN) communication module, or a power line communication module). Among these communication modules, the corresponding communication module can communicate with an external electronic device (104) via a first network (198) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (199) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules can be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can verify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) by using subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (196).

[0039] The wireless communication module (192) can support 5G networks and next-generation communication technologies following the 4G network, such as NR access technology (new radio access technology). The NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and connection of multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency communications)). The wireless communication module (192) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The wireless communication module (192) can support various technologies for securing performance in a high-frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), an external electronic device (e.g., the electronic device (104)), or a network system (e.g., the second network (199)). According to one embodiment, the wireless communication module (192) can support a peak data rate (e.g., 20 Gbps or more) for eMBB realization, a loss coverage (e.g., 164 dB or less) for mMTC realization, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL), or 1 ms or less for round trip) for URLLC realization.

[0040] The antenna module (197) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). In one embodiment, the antenna module (197) may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). In one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (198) or the second network (199), may be selected from the plurality of antennas, for example, by the communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device via the at least one selected antenna. In some embodiments, in addition to the radiator, another component (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module (197).

[0041] According to various embodiments, the antenna module (197) may form a mmWave antenna module. In one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent a first side (e.g., a bottom side) of the printed circuit board and capable of supporting a designated high-frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) disposed on or adjacent a second side (e.g., a top side or a side side) of the printed circuit board and capable of transmitting or receiving signals in the designated high-frequency band.

[0042] At least some of the above components can be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)).

[0043] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) via a server (108) connected to a second network (199). Each of the external electronic devices (102 or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations executed in the electronic device (101) may be executed in one or more of the external electronic devices (102, 104, or 108). For example, when the electronic device (101) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (101) may, instead of or in addition to executing the function or service itself, request one or more external electronic devices to perform the function or at least a part of the service. One or more external electronic devices that receive the request may execute at least a portion of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may process the result as is or additionally and provide it as at least a portion of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used, for example. The electronic device (101) may provide an ultra-low latency service by using distributed computing or mobile edge computing, for example. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server utilizing machine learning and / or a neural network. According to one embodiment, the external electronic device (104) or the server (108) may be included in the second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0044] Electronic devices according to the various embodiments disclosed in this document may take various forms. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. Electronic devices according to the embodiments of this document are not limited to the aforementioned devices.

[0045] The various embodiments of this document and the terminology used therein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly indicates otherwise. In this document, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" can include any one of the items listed together in the corresponding phrase among those phrases, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used merely to distinguish one component from another, and do not limit the components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as "coupled" or "connected" to another (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0046] The term "module" used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or part of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0047] Various embodiments of the present document may be implemented as software (e.g., a program (140)) including one or more instructions stored in a storage medium (e.g., an internal memory (136) or an external memory (138)) readable by a machine (e.g., an electronic device (101)). For example, a processor (e.g., a processor (120)) of the machine (e.g., an electronic device (101)) may call at least one instruction among the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' simply means that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently or temporarily on the storage medium.

[0048] According to one embodiment, the method according to various embodiments disclosed in this document may be provided as a computer program product. The computer program product may be traded between sellers and buyers as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or may be provided through an application store (e.g., Play Store). TM ) or directly between two user devices (e.g., smart phones), online distribution (e.g., downloading or uploading). In the case of online distribution, at least a portion of the computer program product may be at least temporarily stored or temporarily created in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0049] According to various embodiments, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separated and placed in other components. According to various embodiments, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to various embodiments, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

[0050] FIG. 2 is a diagram for explaining the creation of a target UI object according to one embodiment of the present disclosure.

[0051] Referring to FIG. 2, an electronic device (200) (e.g., the electronic device (101) of FIG. 1) is illustrated that generates target UI objects (230) based on UI objects (210) and objects (220) of apps. Operations of the electronic device (200) described in the present disclosure can be performed by executing commands stored in a memory (e.g., the memory (130) of FIG. 1). The commands may be stored in one memory or may be divided and stored in multiple memories. The commands can be executed by one or more processors (e.g., the processor (120) of FIG. 1), and when the commands are executed by one or more processors, they can cause the electronic device to perform operations described in the present disclosure.

[0052] The target UI objects (230) may be personalized UI objects. That is, the target UI objects (230) may be personalized UI objects by including target objects that have been converted so that the objects (220) can represent the properties of the app. The target objects may be generated based on a portion of the appearance of the object (220). The target UI objects (230) may include target objects associated with the performance of the functions of the app as UI objects that have been converted so that they can represent the properties of the app. For example, if the app is a workout app, the target UI object of the workout app may include a running target object so that it can be associated with the function of the app (e.g., workout management). The target UI objects (230) may be displayed on the display module instead of the UI objects (210).

[0053] The electronic device (200) can acquire an image to generate target UI objects (230). The image can include an object (220) for generating the target object. The object (220) can be a user of the electronic device (200), as well as other people, animals, characters, objects, etc. The electronic device (200) can initiate generation of the target UI objects (230) using various methods. Methods for initiating generation of the target UI objects (230) will be described with reference to FIGS. 3 to 5.

[0054] According to one embodiment, when an image includes multiple objects, the electronic device (200) may receive a command from a user to create target UI objects (230) using any one of the multiple objects. According to one embodiment, when an image includes multiple objects, the electronic device (200) may display candidate objects and receive a user's selection of an object (220) for creating target UI objects (230) from among the candidate objects.

[0055] According to one embodiment, the electronic device (200) can acquire a plurality of images including an object (220). The electronic device (200) can generate target UI objects (230) using the object (220) included in the plurality of images.

[0056] An electronic device (200) can extract features from an image. The features may include a first object included in the image, color information of the image, or the background of the image. The features may be referred to as second feature information to distinguish them from the feature information of the app described below (i.e., first feature information).

[0057] An electronic device (200) can extract an object (220) from an image. The electronic device can extract the object (220) from the image using various methods. For example, the electronic device can extract the object (220) from the image using a deep learning-based object extraction model. According to one embodiment, the object (220) can be extracted from the electronic device (200) or from a server (e.g., server (108) of FIG. 1) that communicates with the electronic device (200). When the object (220) is extracted from the server, the electronic device can transmit the image to the server and, in response, receive the object (220) extracted from the image from the server.

[0058] The electronic device (200) may acquire feature information of apps corresponding to UI objects (210) in order to generate target UI objects (230) together with the object (220). The feature information may include one or more of attribute information describing the app, color information included in the UI object of the app, and information about an object included in the UI object of the app. In order to distinguish the object (220) included in the image from the object included in the UI object of the app, they may be referred to as a first object and a second object, respectively. The feature information of the app may be referred to as first feature information in order to distinguish it from a feature portion of the image (i.e., second feature information).

[0059] The electronic device (200) can obtain attribute information through the app market. The attribute information may be information describing the app. The attribute information may include information describing the app, such as the app's category and hashtags. However, this is merely an example, and various information that can describe the app, in addition to the app's category and hashtags, may be included as attribute information. According to one embodiment, the electronic device (200) can obtain attribute information from the app's metadata. For example, the electronic device (200) can obtain attribute information from the manifest file information of an Android app.

[0060] The electronic device (200) can extract color information from the UI object of the app. The color information may refer to the color of the UI object displayed on the display module of the electronic device (200) (e.g., the display module (160) of FIG. 1). The color information may include not only representative colors of the UI object, but also color effects (e.g., gradient effects, grip effect, etc.). The electronic device (200) can obtain the color information using various methods.

[0061] The electronic device (200) can obtain information about a second object included in a UI object. The electronic device (200) can extract the second object from the UI object using various methods. For example, the electronic device can extract the second object from the UI object using a deep learning-based object extraction model. According to one embodiment, the deep learning-based object extraction model can be stored in the memory of the electronic device (200) or stored in a server communicating with the electronic device (200). Accordingly, the second object included in the UI object can be extracted from the electronic device (200) or from a server communicating with the electronic device (200). When the second object is extracted from the server, the electronic device can transmit the UI object to the server and, in response, receive the second object extracted from the UI object from the server.

[0062] The electronic device (200) can determine whether the extracted second object has a human shape. If the extracted second object has a human shape, the electronic device (200) can identify what pose it takes.

[0063] The electronic device (200) can generate a target UI object based on the characteristic information of the object (220) and the app. The electronic device (200) can generate a target UI object based on the characteristic and characteristic information of the object (220). The characteristic of the object (220) may be a face in the case of a human. A method for generating a target UI object when the second object has a human shape and takes a specific pose will be described with reference to FIG. 7. A method for generating a target UI object when the second object does not have a human shape will be further described with reference to FIGS. 6 and 8.

[0064] There may be various methods for generating a target UI object based on the object (220) and feature information. For example, a deep learning-based generative model may output a target UI object using the object (220) and feature information as input. For example, a generative AI may output a target UI object using the feature information of the object (220) and the app as input. The generative AI may be a model trained by an electronic device to generate a target UI object based on the feature information and / or auxiliary information of the object (220), UI objects (210). The generative AI may be stored in the memory of the electronic device (200) or in a server communicating with the electronic device (200). Accordingly, the target UI object may be generated in the electronic device (200) or in a server communicating with the electronic device (200) through the generative AI. When the target UI object is generated in the server, the electronic device may transmit the feature information of the object (220) and the app to the server and receive the target UI object from the server in response.

[0065] The electronic device (200) can generate a target UI object based on auxiliary information as well as object (220) and feature information. A method for generating a target UI object using auxiliary information will be described in FIG. 9.

[0066] In the present disclosure, UI objects (user interface objects) may refer to interface elements that enable interaction between an app and a user, such as an app icon or an app widget. Therefore, for convenience of explanation, if a UI object is described as an app icon, it will be apparent to those skilled in the art that the description of an app icon can also be applied to interface elements such as an app widget. Furthermore, according to one embodiment, UI objects may include interface elements such as a wallpaper, a lock screen, and an always-on display (AOD).

[0067] Additionally, the image objects used to create the target UI object in the present disclosure may be people, animals, characters, objects, etc. For convenience of explanation, the image objects will be assumed to be people, and a method for creating a target UI object based on a human face will be described, for example. However, it will be apparent to those skilled in the art that the creation of a target UI object using a person can also be applied to animals, characters, objects, etc.

[0068] Below, we will explain how to start creating target UI objects (230).

[0069] FIG. 3 and FIG. 4 are drawings for explaining an example of creating a target UI object according to one embodiment of the present disclosure.

[0070] Referring to FIG. 3, an electronic device (300) (e.g., the electronic device (101) of FIG. 1 and the electronic device (200) of FIG. 2) that generates a target icon (330) using an icon generation tool is illustrated. The electronic device (300) may provide an icon generation tool capable of generating a target icon (330). The icon generation tool may be a native application provided by the manufacturer of the electronic device (300) or a third-party application provided by an app market, etc.

[0071] The electronic device (300) may receive from the user an image to be used to generate a target icon (330). According to one embodiment, the image (320) may be an image that includes only a first object (325) (e.g., object (220) of FIG. 2) to be used to generate the target icon (330). According to one embodiment, the image (320) may be an image that is cropped from an original image that includes a plurality of objects to include only the first object (325).

[0072] The image (320) can be reselected. The image (320) can be reselected from among images stored in the electronic device (300). The image (320) can be reselected as an image captured directly by a camera included in the electronic device (300) (e.g., the camera module (180) of FIG. 1). According to one embodiment, the image (320) can be an image automatically selected based on information of an app installed in the electronic device (300). In other words, the image (320) can be an image recommended based on information of an app installed in the electronic device (300). According to one embodiment, the image (320) can be an image automatically selected at random.

[0073] Icons (310) (e.g., UI objects (210) of FIG. 2) may be a list of icons to be personalized based on the first object (325). According to one embodiment, icons (310) may be icons of all apps stored in the electronic device (300). According to one embodiment, icons (310) may be icons selected for personalization from among all apps stored in the electronic device (300).

[0074] The electronic device (300) can generate a target icon (330) when an image (320) and personalized apps are selected. In the present disclosure, only one target icon (330) is illustrated for convenience of explanation, but it is obvious to those skilled in the art that target icons corresponding to the icons (310) are generated.

[0075] The target icon (330) may include a target object (335). The target object (335) may be generated based on the first object (325) and the feature information of the app corresponding to the target object (335) (i.e., the first feature information). In addition, the target icon (330) may be generated based on the first object (325) and the feature information of the app.

[0076] According to one embodiment, the electronic device (300) may obtain a first image (e.g., image (320)). The electronic device (300) may generate a second image using at least a portion of information related to an app that will generate a target UI object and at least a portion of auxiliary information based on at least a portion of the features of the first image. In FIG. 3, the target UI object may be a target icon (330). The electronic device may generate the target UI object based on the second image. In this case, the image (320) may be referred to as a first image to distinguish it from the second image. In other words, the target object described in the present disclosure may be an object included in the second image.

[0077] At this time, the electronic device may generate a target UI object using an artificial intelligence model. The artificial intelligence model may be a model trained to generate a second image using at least some of the information and auxiliary information related to the app that will generate the target UI object based on at least some of the features of the first image, and to generate the target UI object based on the second image.

[0078] In one embodiment, the target UI object may be obtained by reducing or cropping the second image. Alternatively, the target UI object may be obtained by converting the second image to a resolution optimized for the resolution supported by the display of the electronic device.

[0079] According to one embodiment, the electronic device may generate a target UI object that reflects the outline shape (e.g., circle, square, rounded square, etc.) of a theme and / or UI object (e.g., icon) applied to the electronic device.

[0080] In one embodiment, the electronic device may generate a 3D target UI object based on whether the electronic device supports 3D effects. A method for generating a 3D target UI object is further described in FIG. 12.

[0081] According to one embodiment, when generating and storing a target UI object based on a second image, the electronic device may convert the extension of the second image (e.g., jpg, png, etc.) to the extension of the target UI object (e.g., ico).

[0082] Referring to FIG. 4, a drawing is shown for explaining a method of generating target icons (430) (e.g., target icon (330) of FIG. 3) in a gallery app of an electronic device (400) (e.g., electronic device (101) of FIG. 1, electronic device (200) of FIG. 2, and electronic device (300) of FIG. 3).

[0083] In one embodiment, the electronic device (400) may enter a home screen editing mode to create a personalized icon. The electronic device (400) may receive a user input (414) for entering the home screen editing mode. For example, the user input (414) may include an input (e.g., a long press) for selecting a home screen (or an empty area of ​​the screen). It is assumed that the home screen includes at least one UI object (e.g., a health app icon (411), a music app icon (412), a messenger app icon (413), and / or a widget).

[0084] According to one embodiment, the electronic device (400) may display a visual object (417) for icon creation in an area of ​​the home screen editing mode. When the electronic device (400) receives a user input (415) for selecting the visual object (417), the electronic device (400) may obtain a first image (420) (e.g., image (320) of FIG. 3) for icon creation. For example, the electronic device (400) may display a gallery app execution screen to obtain the first image (420). Alternatively, for example, the electronic device (400) may display a submenu including a gallery app execution item or a camera shooting item, and when a user input for selecting a camera shooting item is received, the electronic device (400) may execute a camera app to obtain the first image (420) in real time. Furthermore, according to one embodiment, when the electronic device (400) receives a user input (415) for selecting the visual object (417), the electronic device (400) may provide the icon creation tool of FIG. 3. Hereinafter, a case of displaying the gallery app execution screen will be described.

[0085] Additionally, according to one embodiment, when the electronic device receives a user input (415) for selecting a visual object (417), the electronic device may additionally display a screen (e.g., a list of app icons and / or a list of widgets) for selecting at least one UI object for generating a personalized icon. The electronic device (400) may generate a personalized icon for the at least one selected UI object (e.g., a health app icon (411), a music app icon (412), and a messenger app icon (413)).

[0086] The electronic device (400) can display at least one image stored in the electronic device (400) in the gallery app. The electronic device (400) can select (or acquire) a first image (420) and receive a user input for creating an icon (e.g., selecting an apply button (416) input).

[0087] When receiving a user input, the electronic device (400) may generate at least one personalized target UI object (e.g., a health app icon (431), a music app icon (432), a messenger app icon (433), and / or a widget) based on at least a portion of a feature portion (i.e., second feature information) of the first image (420). The feature portion of the first image (420) may include an object included in the first image (420), color information of the first image, or a background of the first image.

[0088] According to one embodiment, if there is an input operation for a threshold time or longer with respect to the first image (420), the electronic device may display a pop-up window. The pop-up window may provide various options for the selected first image (420). The pop-up window may provide a "Create as Icon" option for the first image (420). If the "Create as Icon" option is selected, the electronic device (400) may generate target icons (430) using the first image (420).

[0089] According to one embodiment, when the "Create as Icon" option is selected, the electronic device may display the icon creation tool of FIG. 3 and create a target icon (430) through the icon creation tool. According to one embodiment, when the "Create as Icon" option is selected, the electronic device (400) may create a target icon (430) in the background and then display only the created target icon (430).

[0090] According to one embodiment, the electronic device (400) can select multiple images from a gallery app. The electronic device (400) can generate a target icon (430) using the selected multiple images. The electronic device (400) can identify whether the multiple images contain a common object. The electronic device (400) can generate the target icon (430) using the common object.

[0091] FIG. 5 is a diagram for explaining the creation of a target UI object when multiple users use an electronic device according to one embodiment of the present disclosure.

[0092] An electronic device (500) (e.g., electronic device (101) of FIG. 1, electronic device (200) of FIG. 2, electronic device (300) of FIG. 3, and electronic device (400) of FIG. 4) may be a device shared by multiple users (e.g., user A and user B). The electronic device (500) may perform biometric recognition and unlock the device based on the biometric recognition results. For example, the electronic device (500) may unlock the device by identifying the user through facial recognition.

[0093] The electronic device (500) can generate personalized target icons (531, 533) using the image acquired for unlocking.

[0094] For example, when user A attempts to unlock, the electronic device (500) can obtain an image (521) of user A (e.g., image (320) of FIG. 3 and image (420) of FIG. 4) in real time for unlocking. The electronic device can unlock if an object included in the image (521) is identified as user A. Simultaneously with unlocking, the electronic device (500) can generate personalized target icons (531) for icons (510) of apps stored in the electronic device (500) (e.g., UI objects (210) of FIG. 2 and icons (310) of FIG. 3) using the image (521) of user A. After unlocking, the electronic device (500) can display the personalized target icons (531) on the background screen. The target icons (531) can include a target object generated based on the image (521) of user A.

[0095] Similarly, when user B attempts to unlock, the electronic device can obtain an image (523) of user B (e.g., image (320) of FIG. 3 and image (420) of FIG. 4) in real time for unlocking. Then, the electronic device (500) can use the image (523) to generate personalized target icons (533) for icons (510) of apps stored in the electronic device (500). The target icons (531) are personalized icons for user A, and the target icons (533) are personalized icons for user B, and thus may be different from each other. That is, the target icons (531) may include target objects (560) generated based on user A, and the target icons (533) may include target objects (570) generated based on user B.

[0096] According to one embodiment, when the electronic device (500) generates personalized target icons (531, 533), it can store the generated target icons (531, 533) and then retrieve and display them on the background screen when the device is unlocked later. In other words, instead of generating new target icons (531, 533) each time the device is unlocked, previously generated target icons can be retrieved. For example, when target icons (531) for user A are generated, the electronic device (500) can store the target icons (531). Then, when user A unlocks the electronic device (500) later, the electronic device (500) can identify that the user unlocking the device is user A, retrieve the target icons (531) for user A, and display them on a display module (e.g., the display module (160) of FIG. 1). At this time, since the target icons (531) have already been created and stored, they can be simply retrieved. Accordingly, if the user who wishes to unlock the electronic device (500) is identified as user A through various biometric recognition methods, as well as facial recognition, the electronic device (500) can retrieve the target icons (531) and display them on the display module.

[0097] According to one embodiment, the electronic device (500) may store personalized target icons only for some apps, retrieve them upon unlocking, and display them on the display module. In other words, target icons may be generated for the remaining apps upon unlocking. Some of the apps to be stored among all target icons may be determined based on the user's selection. Alternatively, some of the apps to be stored among all target icons may be determined to be apps that took a long time to generate target icons. For example, the apps that took a long time to generate target icons may be apps whose generation time exceeds a threshold time.

[0098] According to one embodiment, the electronic device (500) may compare the image of the user acquired at the time of unlocking with the image of the user previously used when generating target icons. If there is a difference between the image of the user acquired at the time of unlocking and the image of the user previously used when generating target icons, the electronic device (500) may regenerate target icons using the image of the user acquired at the time of unlocking. For example, if the hairstyle of user A is long in the image of user A previously used, but the hairstyle of user A is short in the image of user A acquired at the time of unlocking, the electronic device may regenerate target icons (531) using the image of user A acquired at the time of unlocking. The target object (560) included in the regenerated target icons (531) may have a short hairstyle.

[0099] According to one embodiment, the electronic device (500) may regenerate target icons upon the next unlock if a threshold time has elapsed since the target icons were generated. For example, the electronic device may regenerate target icons (531) upon the next unlock if a threshold time (e.g., one month) has elapsed since the target icons (531) for user A were generated.

[0100] Below, we will explain how to create a target UI object.

[0101] FIGS. 6 to 9 are drawings for explaining a generated target UI object according to one embodiment of the present disclosure.

[0102] Referring to FIG. 6, a target UI object (630) (e.g., target icon (330) of FIG. 3 and target icon (430) of FIG. 4) generated based on an image (620) (e.g., image (320) of FIG. 3, image (420) of FIG. 4, image (521) of FIG. 5, and image (523) of FIG. 5) and feature information of an app (i.e., first feature information) is illustrated. The UI object (610) may be an icon for executing an app.

[0103] The image (620) may include a first object (625) (e.g., the object (220) of FIG. 2 and the first object (325) of FIG. 3) used to generate a target UI object (630). An electronic device (e.g., the electronic device (101) of FIG. 1, the electronic device (200) of FIG. 2, the electronic device (300) of FIG. 3, the electronic device (400) of FIG. 4, and the electronic device (500) of FIG. 5) may obtain the image (620) and feature information of the app. The method of obtaining the image (620) and extracting the first object (625) from the image (620) is described above with reference to FIGS. 2 to 5, and thus will not be described again. The method of obtaining the feature information of the app is described above with reference to FIG. 2, and thus will not be described again.

[0104] The feature information of the app may include one or more of attribute information describing the app, color information included in the UI object (610), and information about a second object (615) included in the UI object (610) of the app. For example, referring to FIG. 6, the UI object (610) may be an icon that runs a music player app.

[0105] For example, an electronic device can obtain information describing a music player app, such as information with a music category and hashtags with music and audio. The electronic device can obtain information, such as the color information of the music player app, such as the color that constitutes the background of the app and the color that constitutes the second object (615) included in the app. The electronic device can obtain the second object (615). Accordingly, the electronic device can obtain the second object (615) having a musical note shape. Additionally, the electronic device can determine whether the second object (615) is a human shape. A method for generating a target UI object (630) when the second object (615) is a human shape will be further described in FIG. 7.

[0106] The electronic device can generate a target UI object (630) based on the first object (625) and feature information. The method for generating the target UI object (630) is described above with reference to FIG. 2, so a detailed description thereof will be omitted.

[0107] The target UI object (630) may be a UI object personalized with the first object (625) while representing the characteristics of the app. For example, the target UI object (630) may represent the characteristics of the app (i.e., a music player app) by including the target object (635) (e.g., target object (335) of FIG. 3) of the first object (625) listening to music using a headset.

[0108] According to one embodiment, the target UI object (630) may be generated using only the first object (625) and the attribute information of the app. Since the target UI object (630) must be personalized as the first object (625) while representing the characteristics of the app, the attribute information of the first object (625) and the app may be essential. Accordingly, the target UI object (630) generated based only on the attribute information of the first object (625) and the app may also be personalized as the first object (625) while representing the characteristics of the app. However, the target UI object (630) generated using not only the attribute information of the first object (625) and the app, but also color information and information about the second object (615) may be consistent with the original UI object (610) of the app.

[0109] A target object (635) for appreciating music can be created based on attribute information.

[0110] According to one embodiment, an electronic device may obtain a first image (e.g., image (620)). The electronic device may generate a second image using at least a portion of information related to an app that will generate a target UI object (630) based on at least a portion of the features of the first image and at least a portion of the auxiliary information. At least a portion of the features of the first image may include at least a portion of the first object (625) included in the first image and the color information of the first image or the background of the first image as a feature portion (i.e., second feature information) of the first image. In addition, the feature portion of the first image may include detailed information about the first object (625), such as clothing coordination information of the first object (625). The electronic device may generate a target UI object (630) based on the second image. In this case, the image (620) may be referred to as a first image to distinguish it from the second image. The target object (635) described in the present disclosure may be an object included in the second image.

[0111] According to one embodiment, the electronic device may generate a target UI object (630) using an artificial intelligence model. The artificial intelligence model may be a model trained to generate a second image using at least some of the information and auxiliary information related to the app that will generate the target UI object (630) based on at least some of the features of the first image, and to generate the target UI object based on the second image.

[0112] In one embodiment, the target UI object (630) may be obtained by reducing or cropping the second image. Alternatively, the target UI object (630) may be obtained by converting the second image to a resolution optimized for the resolution supported by the display of the electronic device.

[0113] According to one embodiment, the electronic device may generate a target UI object (630) that reflects the theme and / or the outline shape (e.g., circle, square, rounded square, etc.) of the UI object (610) applied to the electronic device.

[0114] The method of generating a second image and generating a target UI object based on the second image can be equally applied to various embodiments of the present disclosure, and thus a description thereof is omitted below.

[0115] Referring to FIG. 7, an example of creating a target UI object when a second object included in a UI object is in the shape of a human is illustrated.

[0116] Referring to FIG. 7, an image (720) (e.g., an image (320) of FIG. 3, an image (420) of FIG. 4, an image (521) of FIG. 5, an image (523) of FIG. 5, and an image (620) of FIG. 6) and a target UI object (730) (e.g., a target icon (330) of FIG. 3, a target icon (430) of FIG. 4, and a target UI object (630) of FIG. 6) generated based on feature information of the app (i.e., first feature information) are illustrated.

[0117] An electronic device (e.g., an electronic device (101) of FIG. 1, an electronic device (200) of FIG. 2, an electronic device (300) of FIG. 3, an electronic device (400) of FIG. 4, and an electronic device (500) of FIG. 5) can obtain an image (720) and feature information of an app to generate a target UI object (730).

[0118] The feature information of the app may include one or more of the property information of the app, the color information included in the UI object (710) of the app, and the information about the second object (715) included in the UI object (710) of the app (e.g., the second object (615) of FIG. 6).

[0119] The UI object (710) (e.g., the UI object (610) of FIG. 6) may be an icon for executing a health-related app. For example, the electronic device may obtain information describing the health-related app, such as information with a category of health and fitness and hashtags of health, self-management, life coaching, diet, and exercise. The electronic device may obtain a color that constitutes the background of the UI object (710) and a color that constitutes a second object (715) included in the UI object (710) as color information of the health-related app. For example, the electronic device may obtain green as the color that constitutes the background of the UI object (710) and white as the color that constitutes the second object (715). The electronic device may obtain a color effect of the UI object (710) as color information of the health-related app. For example, the electronic device may obtain a gradient effect as the color effect of the app.

[0120] The electronic device can acquire a second object (715). The electronic device can determine whether the second object (715) is a human figure. A human figure may include a character, a silhouette of a human figure, etc. For example, since the second object (715) represents a portion of the external appearance of a running person, the electronic device can determine that the second object (715) is a human figure.

[0121] The electronic device can acquire a target UI object (730) using the first object (725) (e.g., the object (220) of FIG. 2 and the first object (625) of FIG. 6) and the feature information of the app. The target UI object (730) may reflect the feature information of the first object (725) and the app. For example, the background color of the target UI object (730) may be green and a gradient effect may be applied. The pose of the target object (735) included in the target UI object (730) (e.g., the target object (335) of FIG. 3 and the target object (635) of FIG. 6) may correspond to a portion of the appearance of a running person of the second object (715). The face of the target object (735) may correspond to a portion of the appearance of the object (725). That is, the target object (735) can be generated by synthesizing a portion of the appearance of the first object (725) and a portion of the appearance of the second object (715), and can assume a pose corresponding to a specific pose assumed by the second object (715).

[0122] In one embodiment, the electronic device may reference the usage history information of the app rather than the shape of the second object (715). The electronic device may use the usage history information of the app to create a target object (735). For example, the electronic device may reference the usage history information of a health-related app. Based on the results of referencing the usage history of the health-related app (e.g., the user's favorite sport (e.g., tennis)), the electronic device may create a personalized target object for playing tennis.

[0123] Also, referring to FIG. 7, an image (750) (e.g., an image (320) of FIG. 3, an image (420) of FIG. 4, an image (521) of FIG. 5, an image (523) of FIG. 5, and an image (620) of FIG. 6) and a target UI object (760) generated based on feature information of the app (e.g., a target icon (330) of FIG. 3, a target icon (430) of FIG. 4, and a target UI object (630) of FIG. 6) are illustrated.

[0124] The electronic device can obtain image (750) and feature information of the app to generate a target UI object (760).

[0125] The UI object (740) (e.g., the UI object (610) of FIG. 6) may be an icon for executing a game app. For example, the electronic device may obtain information describing the game app, such as information with categories of games and strategies and hashtags of role-playing, adventure, and multiplayer. The electronic device may obtain a color constituting the background of the UI object (740) and a color constituting the second object (745) (e.g., the second object (615) of FIG. 6) included in the UI object (740) as color information of the game app. For example, the electronic device may obtain purple as the color constituting the background, and yellow and apricot as the colors constituting the second object (745). The electronic device may obtain a color effect of the UI object (740) as color information of the game app. For example, the electronic device may obtain no effect as the color effect of the game app.

[0126] The electronic device can acquire a second object (745). The electronic device can determine whether the second object (745) is human-shaped. For example, since the second object (745) represents a portion of the external appearance of a screaming person, the electronic device can determine that the second object (745) is human-shaped.

[0127] The electronic device can acquire a target UI object (760) using the first object (755) (e.g., the object (220) of FIG. 2 and the first object (625) of FIG. 6) and the feature information of the app. The target UI object (760) may reflect the feature information of the first object (755) and the app. For example, the background color of the target UI object (760) may be purple and there may be no color effect. The pose of the target object (765) included in the target UI object (760) (e.g., the target object (335) of FIG. 3 and the target object (635) of FIG. 6) may correspond to a portion of the appearance of a screaming person corresponding to the second object (745). The face of the target object (765) may correspond to a portion of the appearance of the first object (755). That is, the target object (765) can be generated by synthesizing a portion of the appearance of the first object (755) and a portion of the appearance of the second object (745), and can assume a pose corresponding to a specific pose assumed by the second object (745).

[0128] Referring to FIG. 8, the creation of a target UI object will be described when the second object is not identified as a human shape.

[0129] Referring to FIG. 8, an image (820) (e.g., an image (320) of FIG. 3, an image (420) of FIG. 4, an image (521) of FIG. 5, an image (523) of FIG. 5, an image (620) of FIG. 6, an image (720) of FIG. 7, and an image (750) of FIG. 7) and a target UI object (830) (e.g., a target icon (330) of FIG. 3, a target icon (430) of FIG. 4, a target UI object (630) of FIG. 6, a target UI object (730) of FIG. 7, and a target UI object (760) of FIG. 7) generated based on feature information of the app are illustrated.

[0130] An electronic device (e.g., an electronic device (101) of FIG. 1, an electronic device (200) of FIG. 2, an electronic device (300) of FIG. 3, an electronic device (400) of FIG. 4, and an electronic device (500) of FIG. 5) can obtain an image (820) and feature information of an app to generate a target UI object (830).

[0131] The feature information of the app may include one or more of the attribute information of the app, the color information included in the UI object (810) of the app, and the information on the second object (815) included in the UI object (810) of the app (e.g., the second object (615) of FIG. 6, the second object (715) of FIG. 7, and the second object (745) of FIG. 7).

[0132] The UI object (810) (e.g., the UI object (610) of FIG. 6, the UI object (710) of FIG. 7, and the UI object (740) of FIG. 7) may be an icon for executing a messenger app. For example, the electronic device may obtain information describing the messenger app, such as information with a category of social networking and hashtags of chat, messenger, and communication. The electronic device may obtain a color constituting the background of the UI object (810) and a color constituting the second object (815) included in the UI object (810) using color information of the messenger app. For example, the electronic device may obtain yellow as the color constituting the background, and brown as the color constituting the second object (815). The electronic device may obtain a color effect of the UI object (810) using color information of the messenger app. For example, the electronic device may obtain no effect as the color effect of the app.

[0133] The electronic device can acquire a second object (815). The electronic device can determine whether the second object (815) is a human-shaped object. For example, the electronic device can determine that the second object (815) is a speech bubble-shaped object and not a human-shaped object, and the electronic device can determine that the second object (815) is a logo. The electronic device can determine that the second object (815) occupies more than 50% of the UI object (810) in the speech bubble shape.

[0134] The electronic device can obtain a target UI object (830) using the first object (825) (e.g., the object (220) of FIG. 2, the first object (625) of FIG. 6, the first object (725) of FIG. 7, and the first object (755) of FIG. 7) and the feature information of the app. The target UI object (830) may reflect the feature information of the first object (825) and the app. For example, the background color of the target UI object (830) may be yellow and there may be no color effect. The target object (835) included in the target UI object (830) (e.g., the target object (335) of FIG. 3, the target object (635) of FIG. 6, the target object (735) of FIG. 7, and the target object (765) of FIG. 7) may be generated based on the attribute information and a portion of the appearance of the first object (825). In other words, the target object (835) may be an object that represents property information of the app by using a portion of the appearance of the first object (825). For example, the target object (835) may be created as an image of a person speaking with their hands gathered near their mouth to represent properties of the app, such as communication and chatting. The face of the target object (835) may correspond to a portion of the appearance of the first object (825). In addition, the target UI object (830) may include a second object (815) as well as the target object (835). The second object (833) of the target UI object (830) may have a different size or arrangement than the second object (815) of the UI object (810).

[0135] Also, referring to FIG. 8, an image (850) (e.g., an image (320) of FIG. 3, an image (420) of FIG. 4, an image (521) of FIG. 5, an image (523) of FIG. 5, an image (620) of FIG. 6, an image (720) of FIG. 7, and an image (750) of FIG. 7) and a target UI object (860) generated based on feature information of the app (e.g., a target icon (330) of FIG. 3, a target icon (430) of FIG. 4, a target UI object (630) of FIG. 6, a target UI object (730) of FIG. 7, and a target UI object (760) of FIG. 7) are illustrated.

[0136] An electronic device (e.g., an electronic device (101) of FIG. 1, an electronic device (200) of FIG. 2, an electronic device (300) of FIG. 3, an electronic device (400) of FIG. 4, and an electronic device (500) of FIG. 5) can obtain an image (850) and feature information of an app to generate a target UI object (860).

[0137] The feature information of the app may include one or more of the attribute information of the app, the color information included in the UI object (840) of the app, and the information on the second object (845) included in the UI object (840) of the app (e.g., the second object (615) of FIG. 6, the second object (715) of FIG. 7, and the second object (745) of FIG. 7).

[0138] The UI object (840) (e.g., the UI object (610) of FIG. 6, the UI object (710) of FIG. 7, and the UI object (740) of FIG. 7) may be an icon for executing a video streaming app. For example, the electronic device may obtain information describing the video streaming app, such as information with a category of entertainment and hashtags of movies and dramas. The electronic device may obtain a color constituting the background of the UI object (840) and a color constituting the second object (845) included in the UI object (840) as color information of the video streaming app. For example, the electronic device may obtain black as the color constituting the background, and red as the color constituting the second object (845). The electronic device may obtain a color effect of the UI object (840) as color information of the video streaming app. For example, the electronic device may obtain no effect as the color effect of the app.

[0139] The electronic device can acquire a second object (845). The electronic device can determine whether the second object (845) is human-shaped. For example, the electronic device can determine that the second object (845) is not human-shaped because it is in the shape of a large alphabet N, and the electronic device can determine that the second object (845) is a logo.

[0140] The electronic device can obtain a target UI object (860) using the first object (855) (e.g., the object (220) of FIG. 2, the first object (625) of FIG. 6, the first object (725) of FIG. 7, and the first object (755) of FIG. 7) and the feature information of the app. The target UI object (860) may reflect the feature information of the first object (855) and the app. For example, the background color of the target UI object (860) may be black and there may be no color effect. The target object (865) included in the target UI object (860) (e.g., the target object (335) of FIG. 3, the target object (635) of FIG. 6, the target object (735) of FIG. 7, and the target object (765) of FIG. 7) may be generated based on the attribute information and a portion of the appearance of the first object (855). In other words, the target object (865) may be an object that represents the property information of the app by using a part of the appearance of the first object (855). For example, the target object (865) may be created as an image of a person holding popcorn and watching a video to represent the property of an app such as a movie or drama. In addition, the target UI object (860) may include a second object (845) as well as the target object (865). The second object (863) of the target UI object (860) may have a different size or arrangement from the second object (845) of the UI object (840).

[0141] In one embodiment, an electronic device may reference usage history information of an app. The electronic device may use the usage history information of the app to create target objects (835, 865). The usage history information of the app may be included in the auxiliary information. For example, the electronic device may reference usage history information of a video streaming app. The electronic device may create a target object (865) related to a result of referencing the usage history of the video streaming app (e.g., a video (e.g., a movie) recently watched by the user). For example, the target object (865) may be related to a poster image of a movie recently watched by the user.

[0142] According to one embodiment, if the electronic device determines that the second object (815, 845) is not a human shape, the electronic device may remove the second object (815, 845) when generating the target UI object (830, 860). For example, referring to FIG. 6, the target UI object (630) may not include the second object (615). However, in this case, the target UI object (830, 860) may only include a portion of the appearance of the first object (825, 855) and the target object (835, 865) generated based on the attribute information of the app.

[0143] Referring to FIG. 9, a target UI object generated based on auxiliary information as well as feature information of the first object and app is illustrated.

[0144] The method of an electronic device acquiring an image (920) (e.g., an image (320) of FIG. 3, an image (420) of FIG. 4, an image (521) of FIG. 5, an image (523) of FIG. 5, an image (620) of FIG. 6, an image (720) of FIG. 7, an image (750) of FIG. 7, an image (820) of FIG. 8, and an image (850) of FIG. 8) and feature information (i.e., first feature information) of an app, and generating a target UI object (930) (e.g., a target icon (330) of FIG. 3, a target icon (430) of FIG. 4, a target UI object (630) of FIG. 6, a target UI object (730) of FIG. 7, a target UI object (760) of FIG. 7, a target UI object (830) of FIG. 8, and a target UI object (860) of FIG. 8)) based on the image and the feature information of the app is omitted as described above.

[0145] The electronic device may further receive auxiliary information (940). The auxiliary information (940) may refer to information other than the characteristic information of the app. For example, the auxiliary information (940) may include various types of information, including seasonal information, regional information, and weather information. In addition, the auxiliary information (940) may include information received from other apps, information received from an external server, and information received from the terminal system. The information received from other apps may include information received from apps linked to the app that wants to create the target UI object (930) and / or from apps that have been granted permission by the app that wants to create the target UI object (930). The information received from the external server may include information received from a server that services the app that wants to create the UI object (930) and / or information about the app that wants to create the UI object (930) received from an app store. The information received from the terminal system may include regional information and / or carrier information obtained from subscriber identification module (SIM) information. Information received from the terminal system may include location information received from a global positioning system (GPS).

[0146] According to one embodiment, the auxiliary information (940) may be obtained from an app related to an app that attempts to create a target UI object (930). The app related to the app that attempts to create a target UI object (930) may be an app that is linked to the app that attempts to create the target UI object (930). For example, if the app that attempts to create a target UI object is a workout app and the workout app is linked to a weather app, weather information may be obtained from the weather app in order to create a target UI object for the workout app. In addition, the app related to the app that attempts to create a target UI object (930) may be an app that has been granted access to the app that attempts to create the target UI object (930). For example, let's assume that the app that attempts to create a target UI object (930) is a music player app and that the weather app and the map app have been granted access to the music player app. In this case, the target UI object (930) may be generated based on not only the image (920) and the feature information of the app, but also seasonal information, regional information, and weather information. For example, a target object (935) reflecting seasonal information such as winter and Christmas (e.g., target object (335) of FIG. 3, target object (635) of FIG. 6, target object (735) of FIG. 7, target object (765) of FIG. 7, target object (835) of FIG. 8, and target object (865) of FIG. 8) may be created. In addition, the electronic device may provide a snowfall effect to the target UI object (930) by additionally reflecting weather information (e.g., cold and snowy weather).

[0147] In one embodiment, the auxiliary information (940) may be information entered by the user via a prompt. For example, the electronic device may enter seasonal information, such as "Christmas," via the prompt. In response, the electronic device may generate a target object (935) reflecting the seasonal information.

[0148] FIG. 10 is a diagram illustrating sharing of a generated target UI object according to one embodiment of the present disclosure.

[0149] Referring to FIG. 10, examples of electronic devices (e.g., electronic device (101) of FIG. 1, electronic device (200) of FIG. 2, electronic device (300) of FIG. 3, electronic device (400) of FIG. 4, and electronic device (500) of FIG. 5) are shown, including a smartphone (1000), a tablet PC (1010), a smart watch (1020), and a head mounted display (HMD) (1030).

[0150] The smartphone (1000), tablet PC (1010), smart watch (1020), and HMD (1030) may be electronic devices of the same user. In other words, the smartphone (1000), tablet PC (1010), smart watch (1020), and HMD (1030) may be logged in with the same user's account and synchronized via a network (1040) (e.g., the second network (199) of FIG. 1 ).

[0151] Accordingly, if any one of the electronic devices illustrated in FIG. 10 creates target UI objects according to the operating methods described above in FIGS. 2 to 9, the target UI objects can be shared with the remaining electronic devices.

[0152] For example, if a smartphone (1000) creates target UI objects according to the operation methods described in FIGS. 2 to 9, the created target UI objects can be shared with a tablet PC (1010), a smart watch (1020), and an HMD (1030). As a result, the tablet PC (1010), the smart watch (1020), and the HMD (1030) can also display the target UI objects.

[0153] If a modification of a target UI object occurs in one of the multiple electronic devices being synchronized, the modified target UI object can be shared with the remaining electronic devices via the network (1040).

[0154] FIG. 11 is a diagram illustrating target UI objects with different degrees of personalization according to one embodiment of the present disclosure.

[0155] According to one embodiment, an electronic device (e.g., electronic device (101) of FIG. 1, electronic device (200) of FIG. 2, electronic device (300) of FIG. 3, electronic device (400) of FIG. 4, and electronic device (500) of FIG. 5) may be a foldable electronic device.

[0156] The electronic device can generate target UI objects (e.g., target icon (330) of FIG. 3, target icon (430) of FIG. 4, target UI object (630) of FIG. 6, target UI object (730) of FIG. 7, target UI object (760) of FIG. 7, target UI object (830) of FIG. 8, target UI object (860) of FIG. 8, and target UI object (930) of FIG. 9) with different degrees of personalizedness depending on the location of the display. The location of the display can be included in the auxiliary information.

[0157] The main display (1100) may be a display that the user manipulates more than the cover display (1120). The cover display (1120) may be a display that is exposed to the outside when the foldable electronic device is folded, thereby allowing people other than the user to view it.

[0158] Accordingly, the electronic device can determine the degree of personalization of the target UI object (1110) displayed on the main display (1100) to be higher than the degree of personalization of the target UI object (1130) displayed on the cover display (1120). The higher the degree of personalization, the more similar the target objects (1113, 1133) included in the target UI objects (1110, 1130) may be to the objects of the images used to generate the target UI objects (1110, 1130). For example, if the target UI object (1110) includes a target object (1113) that has the user's face and is running, the target object (1130) may simply include a target object (1133) that has a human shape running. However, the user's face may be diverse, such as not only the owner of the electronic device but also a favorite celebrity of the owner of the electronic device.

[0159] According to one embodiment, an electronic device may set a different degree of personalization depending on the space in which the target UI object is displayed. Information regarding the space in which the target UI object is displayed may be included in the auxiliary information. The electronic device may determine a higher degree of personalization for a target UI object displayed in a private space (e.g., an app drawer) than for a target UI object displayed in a relatively public space (e.g., a home screen).

[0160] In one embodiment, target UI objects with different degrees of personalization may be displayed depending on whether the user has been authenticated. For example, in a financial app, the target UI object for an app that has undergone user authentication may include a more personalized target object than the target UI object for an app that has not. The authentication status may be included in the auxiliary information.

[0161] FIG. 12 is a drawing for explaining a 3D target UI object according to one embodiment of the present disclosure.

[0162] According to one embodiment, an electronic device (e.g., the electronic device (101) of FIG. 1, the electronic device (200) of FIG. 2, the electronic device (300) of FIG. 3, the electronic device (400) of FIG. 4, and the electronic device (500) of FIG. 5) may be a head mounted display (HMD). If the electronic device is an HMD, the electronic device may include a lenticular lens structure and a parallax barrier structure. If the electronic device is an HMD, the electronic device may include a 3D display (1200) that provides a 3D effect. Whether the electronic device includes the 3D display (1200) may be included in the auxiliary information. If the electronic device includes the 3D display (1200), the electronic device may generate a 3D target UI object (1210).

[0163] The electronic device can enable the left-eye image (1220) and the right-eye image (1230) to be recognized as the user's left and right eyes, respectively. Accordingly, when the user views the 3D display (1200) from the front at a predetermined distance from the 3D display (1200), the user can perceive a 3D stereoscopic effect through the binocular illusion.

[0164] The 3D target UI object (1210) may include multiple layers. For example, the 3D target UI object (1210) may include at least one of a background layer, a target object layer, an accessory layer (not shown), and / or an effect layer (not shown). Each layer of the 3D target UI object (1210) may be spaced apart from each other when the user looks at the 3D display (1200) from the front (i.e., based on the Z axis), and when the user's gaze information stops at the 3D target UI object (1210) for a certain period of time, one of the multiple layers may be focused or highlighted.

[0165] FIG. 13 is a diagram for explaining a case where a UI object according to one embodiment of the present disclosure is a widget.

[0166] An electronic device (e.g., an electronic device (101) of FIG. 1, an electronic device (200) of FIG. 2, an electronic device (300) of FIG. 3, an electronic device (400) of FIG. 4, and an electronic device (500) of FIG. 5) can generate a personalized target UI object (1320) (e.g., a target icon (330) of FIG. 3, a target icon (430) of FIG. 4, a target UI object (630) of FIG. 6, a target UI object (730) of FIG. 7, a target UI object (760) of FIG. 7, a target UI object (830) of FIG. 8, a target UI object (860) of FIG. 8, a target UI object (930) of FIG. 9, a target UI object (1110) of FIG. 11, and a target UI object (1130) of FIG. 11)) for not only an app icon but also a widget (1310).

[0167] The electronic device can generate a second image based on the feature portion (i.e., second feature information) of the first image (1300) (e.g., image (320) of FIG. 3, image (420) of FIG. 4, image (521) of FIG. 5, image (523) of FIG. 5, image (620) of FIG. 6, image (720) of FIG. 7, image (750) of FIG. 7, image (820) of FIG. 8, image (850) of FIG. 8, and image (920) of FIG. 9) and feature information of the app (i.e., first feature information), and generate a target UI object (1320) based on the second image. In FIG. 13, the feature portion of the first image (1300) can include clothing coordination information of user A. In FIG. 13, the feature information of the app may include weather widget information (e.g., clear weather information and / or location information) and / or auxiliary information (e.g., season information and / or user A's biometric information (e.g., body temperature)). Accordingly, the electronic device may generate a second image of user A walking in clear weather and generate a personalized target UI object (1320) based on the second image.

[0168] In other words, the second image may be a personalized second image of User A of the first image (1300) with an outfit coordination that matches the weather / season / body temperature information. For example, the second image may include User A with a warm outfit coordination based on cold weather and User A's low body temperature, User A with light clothing based on clear weather and User A's high body temperature, and / or User A with a Christmas-style outfit coordination based on Christmas season information.

[0169] It will be apparent to those skilled in the art that the present disclosure can be applied not only to the app's icons and widgets (1310), but also to various visual UI objects (e.g., wallpaper, lock screen, AOD, and color skins, etc.).

[0170] According to one embodiment, an electronic device can update visual UI objects, such as app icons, widgets, wallpapers, lock screens, and AODs, in the background. For example, if the electronic device obtains Christmas season information as auxiliary information through a weather app and / or a calendar app, the electronic device can update existing target UI objects with target UI objects that reflect the Christmas information and recommend them to the user. The electronic device can allow the user to check the updated target UI objects through various formats, such as a pop-up window and / or a notification bar. For example, the electronic device can provide a pop-up window that includes a guide phrase (e.g., "I made an app icon for Christmas. Would you like to check it out?") that induces confirmation of the updated target UI objects. The electronic device can display the updated target UI objects according to a user input selecting confirmation or apply the updated target UI objects according to the user's selection.

[0171] FIG. 14 is a flowchart for explaining an operation method of an electronic device according to one embodiment of the present disclosure.

[0172] The operations described below may be performed sequentially, but are not necessarily performed sequentially. For example, the order of the operations may be changed, and at least two operations may be performed in parallel. The operations illustrated in FIG. 14 may be performed by at least one component of an electronic device (e.g., the electronic device (101) of FIG. 1 , the electronic device (200) of FIG. 2 , the electronic device (300) of FIG. 3 , the electronic device (400) of FIG. 4 , and the electronic device (500) of FIG. 5 ).

[0173] In operation (1410), the electronic device can acquire an image (e.g., image (320) of FIG. 3, image (420) of FIG. 4, image (521) of FIG. 5, image (523) of FIG. 5, image (620) of FIG. 6, image (720) of FIG. 7, image (750) of FIG. 7, image (820) of FIG. 8, image (850) of FIG. 8, image (920) of FIG. 9, and first image (1300) of FIG. 13).

[0174] In operation (1420), the electronic device can obtain first characteristic information of an app that will generate a target UI object (e.g., target icon (330) of FIG. 3, target icon (430) of FIG. 4, target UI object (630) of FIG. 6, target UI object (730) of FIG. 7, target UI object (760) of FIG. 7, target UI object (830) of FIG. 8, target UI object (860) of FIG. 8, target UI object (930) of FIG. 9, target UI object (1110) of FIG. 11, and target UI object (1130) of FIG. 11).

[0175] In operation (1430), the electronic device can extract second feature information from the image.

[0176] In operation (1440), the electronic device can generate a target UI object related to the app based on the first feature information and the second feature information.

[0177] Since the matters described above through FIGS. 1 to 13 are applied to each operation illustrated in FIG. 14, a more detailed description is omitted.

[0178] FIG. 15 is a block diagram of a generative artificial intelligence system according to various embodiments.

[0179] The User Query / Response Interface can receive user input. User input can be in the form of natural language, images, and / or videos. Context information can also be transmitted when the user input is transmitted. Context information can include various additional information at the time of user input. For example, it can include information about the application the user is currently using or the user's location. Furthermore, user input can be a mixture of natural language, images, sounds, and context information. Furthermore, user input can be in the form of non-natural language, such as selecting a menu. The User Query / Response Interface can output the results of a generative artificial intelligence system to the user. The output can be in the form of natural language or specific content, and can also be provided in the form of an action requested by the user. The User Query Interface can output the results of a generative artificial intelligence system to the user. The output can be in the form of natural language or specific content, and can also be provided in the form of an action requested by the user.

[0180] The AI ​​framework can receive user input and coordinate and control each component necessary to carry out the user's intention based on the user's query.

[0181] User input received from the User Query / Response Interface can be sent to the Prompt design component. The Prompt design component can be used to generate prompts suitable for inputting the user input into large language models (LLMs) or large multimodal models (LMMs). The Prompt design component can be an AI component that uses a machine learning algorithm or a neural network to develop better prompts over time. The Prompt design component can access knowledge components (i.e., knowledge repositories) containing user preference data, a prompt library, and prompt examples based on the user input to generate prompts, and pass the generated prompts to the LLMs or LMMs.

[0182] The API / Plugins management component can communicate with external information when additional information is requested when user input is passed as input to a generative model. The API / Plugins management component establishes a channel for communication with external entities within the AI ​​Interface via APIs, enabling access to various data sources. Furthermore, if an application or service needs to perform an action that ultimately implements user input, rather than an intermediate result, the API / Plugins management component can request that action via an application programming interface (API). Information obtained from external sources can be used to generate prompts in the Prompt design component alongside user input, or can be passed as input to the generative model.

[0183] The Refiner component (i.e., the output modification component) can fine-tune the output from a generative model. For example, the Refiner component can verify that the content generated by the LLM and / or LMM is not irrelevant, biased, or harmful. Furthermore, the Refiner component can determine the degree to which the output matches the user's desired outcome and, if necessary, initiate additional processing. The Refiner component can also configure and provide hints to the user to avoid undesirable output.

[0184] Generative AI models generally refer to artificial intelligence neural networks that generate new forms of data based on user input. Generative AI models may include models that generate images and / or models that generate language. Representative models for generating images include generative adversarial networks (GANs) and variational autoencoders (VAEs), as well as diffusion-based generative models that use VAE and transformer architectures. Language-generating models are models trained to statistically generate the most appropriate output based on input values, and representative examples include models such as CHAT-GPT 3 and CHAT-GPT 4. There are also LMMs that can recognize various types of data input, such as text, images, and voice, and generate new data corresponding to them.

[0185] According to one embodiment, the second feature information may include at least one of the first object (e.g., the object (220) of FIG. 2, the first object (625) of FIG. 6, the first object (725) of FIG. 7, the first object (755) of FIG. 7, the first object (825) of FIG. 8, and the first object (855) of FIG. 8), color information used in the image, and the background of the image.

[0186] In one embodiment, the target UI object is associated with the performance of a function of the app and can be displayed in the display module instead of the UI object of the app.

[0187] According to one embodiment, the first feature information may include one or more of attribute information describing the app, color information included in a UI object of the app (e.g., UI object (610) of FIG. 6, UI object (710) of FIG. 7, UI object (740) of FIG. 7, UI object (810) of FIG. 8, and UI object (840) of FIG. 8), and information about a second object included in a UI object of the app (e.g., second object (615) of FIG. 6, second object (715) of FIG. 7, second object (745) of FIG. 7, second object (815) of FIG. 8, and second object (845) of FIG. 8).

[0188] According to one embodiment, the operation of generating a target UI object may include generating a target UI object based on the first object and the second object of the second feature feature information, when the first feature information includes information about a second object, and the second object is identified as a human shape and takes a specific pose.

[0189] According to one embodiment, the target UI object is generated by synthesizing a portion of the appearance of a first object and a portion of the appearance of a second object, and may include a target object (e.g., target object (335) of FIG. 3, target object (635) of FIG. 6, target object (735) of FIG. 7, target object (765) of FIG. 7, target object (835) of FIG. 8, target object (865) of FIG. 8, and target object (935) of FIG. 9) that assumes a pose corresponding to a specific pose.

[0190] According to one embodiment, the operation of generating a target UI object may generate the target UI object based on attribute information and / or the second object, if the first feature information includes information about a second object and the second object is not identified as a human shape.

[0191] According to one embodiment, the target UI object may include only the target object generated based on attribute information and some of the appearance of the first object, or may include the target object and the second object.

[0192] According to one embodiment, the method of operating the electronic device may further include an operation of obtaining auxiliary information (e.g., auxiliary information (940) of FIG. 9) for generating a target UI object. According to one embodiment, the operation of generating the target UI object may generate the target UI object based on the first object of the second feature section, the first feature information, and the auxiliary information.

[0193] In one embodiment, the auxiliary information may be obtained from other apps installed on the electronic device or may be information input by a user of the electronic device via a prompt.

[0194] According to one embodiment, the method of operating an electronic device may further include an action of sharing a target UI object with one or more electronic devices of a user that are synchronized with the electronic device.

[0195] According to one embodiment, a computer-readable storage medium can store one or more programs including instructions capable of executing any one of the above-described operating methods.

[0196] According to one embodiment, an electronic device may include a memory (e.g., memory 130 of FIG. 1) that stores instructions. The electronic device may include a processor (e.g., processor 120 of FIG. 1) that executes the instructions. The instructions, when executed by the processor, may cause the electronic device to obtain an image. The instructions, when executed by the processor, may cause the electronic device to obtain first feature information of an app for generating a target UI object. The instructions, when executed by the processor, may cause the electronic device to extract second feature information from the image. The instructions, when executed by the processor, may cause the electronic device to generate a target UI object associated with the app based on the first feature information and the second feature information.

[0197] According to one embodiment, the second feature information may include at least one of the first object, color information used in the image, and the background of the image.

[0198] In one embodiment, the target UI object is associated with the performance of a function of the app and can be displayed in the display module instead of the UI object of the app.

[0199] According to one embodiment, the first feature information may include one or more of attribute information describing the app, color information included in a UI object of the app, and information about a second object included in a UI object of the app.

[0200] According to one embodiment, the instructions, when executed by the processor, may cause the electronic device to generate a target UI object based on the first object and the second object of the second feature information, if the first feature information includes information about a second object, and the second object is identified as a human shape and assumes a specific pose.

[0201] According to one embodiment, the target UI object may be generated by synthesizing a portion of the appearance of a first object and a portion of the appearance of a second object, and may include a target object that assumes a pose corresponding to a specific pose.

[0202] According to one embodiment, the instructions, when executed by the processor, may cause the electronic device to generate a target UI object based on the attribute information and / or the second object, if the first feature information includes information about a second object and the second object is not identified as a human figure.

[0203] According to one embodiment, the target UI object may include only the target object generated based on attribute information and some of the appearance of the first object, or may include the target object and the second object.

[0204] According to one embodiment, the instructions, when executed by the processor, may cause the electronic device to obtain auxiliary information about the app. The instructions, when executed by the processor, may cause the electronic device to generate a target UI object based on the first object, the first feature information, and at least a portion of the auxiliary information.

[0205] In one embodiment, the auxiliary information may be information obtained from apps associated with the app or information entered by a user of the electronic device via a prompt.

[0206] According to one embodiment, the instructions, when executed by the processor, may cause the electronic device to share a target UI object with one or more electronic devices of a user that are synchronized with the electronic device.

[0207] According to one embodiment, an electronic device may include a memory that stores instructions. The electronic device may include a processor that executes the instructions. The instructions, when executed by the processor, may cause the electronic device to obtain at least one first image. The instructions, when executed by the processor, may cause the electronic device to generate a second image using at least some of information and auxiliary information related to an app that will generate at least one target UI object based on at least some of the features of the at least one first image. The instructions, when executed by the processor, may cause the electronic device to generate at least one target UI object associated with performing a function of the app based on the second image.

[0208] According to one embodiment, the instructions, when executed by the processor, may cause the electronic device to generate at least one target UI object using an artificial intelligence model. The artificial intelligence model may be a model trained to generate a second image using at least some of the information and auxiliary information related to the app that generates at least one target UI object based on at least some of the features of at least one first image, and to generate at least one target UI object based on the second image.

[0209] According to one embodiment, a method of operating an electronic device may include an operation of acquiring at least one first image. The method of operating the electronic device may include an operation of generating a second image using at least a portion of information and auxiliary information related to an app that generates at least one target UI object based on at least a portion of features of the at least one first image. The method of operating the electronic device may include an operation of generating at least one target UI object associated with performing a function of the app based on the second image.

[0210] According to one embodiment, the instructions, when executed by the processor, may cause the electronic device to obtain a first image. The instructions, when executed by the processor, may cause the electronic device to generate a first prompt for generating a second image using an AI algorithm based on the first image. The instructions, when executed by the processor, may cause the electronic device to generate a second prompt, wherein at least a portion of the first prompt is modified, using at least a portion of first application-related information installed on the electronic device and first auxiliary information acquired using the electronic device. The instructions, when executed by the processor, may cause the electronic device to generate a second image using an AI algorithm based on the second prompt. The instructions, when executed by the processor, may cause the electronic device to generate at least one UI object related to the execution of a function of the application using at least a portion of the second image.

[0211] A method of operating an electronic device according to one embodiment may include an operation of acquiring a first image. The method of operating the electronic device may include an operation of generating a first prompt for generating a second image using an AI algorithm based on the first image. The method of operating the electronic device may include an operation of generating a second prompt in which at least a portion of the first prompt is modified using at least a portion of first application-related information installed on the electronic device and first auxiliary information acquired using the electronic device. The method of operating the electronic device may include an operation of generating a second image using an AI algorithm based on the second prompt. The method of operating the electronic device may include an operation of generating at least one UI object related to execution of a function of an application using at least a portion of the second image.

[0212] The embodiments of the present invention disclosed in this specification and drawings are merely specific examples presented to easily explain the technical contents according to the embodiments of the present invention and to help understand the embodiments of the present invention, and are not intended to limit the scope of the embodiments of the present invention. Therefore, the scope of the various embodiments of the present invention should be interpreted as including all changes or modified forms derived based on the technical ideas of the various embodiments of the present invention in addition to the embodiments disclosed herein.

Claims

1. In electronic devices (101, 200, 300, 400, 500), Memory (130) for storing commands; and Processor (120) executing the above commands Including, The above instructions, when executed by the processor (120), cause the electronic device (101, 200, 300, 400, 500) to: Obtaining an image (320, 420, 521, 523, 620, 720, 750, 820, 850, 920, 1300), obtaining first feature information of an app to generate a target UI object (330, 430, 630, 730, 760, 830, 860, 930, 1110, 1130), extracting second feature information from the image (320, 420, 521, 523, 620, 720, 750, 820, 850, 920, 1300), and generating the target UI object (330, 430, 630, 730, 760, 1110, 1130) related to the app based on the first feature information and the second feature information. to generate 830, 860, 930, 1110, 1130). Electronic devices (101, 200, 300, 400, 500).

2. In paragraph 1, The above second characteristic information is, The first object, at least one of color information used in the image (320, 420, 521, 523, 620, 720, 750, 820, 850, 920, 1300) and the background of the image, Electronic devices (101, 200, 300, 400, 500).

3. In either of paragraphs 1 and 2, The above target UI objects (330, 430, 630, 730, 760, 830, 860, 930, 1110, 1130) are Associated with the performance of the function of the above app, and displayed on the display module (160) instead of the UI object (610, 710, 740, 810, 840) of the above app, Electronic devices (101, 200, 300, 400, 500).

4. In any one of paragraphs 1 to 3, The above first characteristic information is, Including at least one of attribute information describing the app, color information included in a UI object (610, 710, 740, 810, 840) of the app, and information about a second object (615, 715, 745, 815, 845) included in a UI object (610, 710, 740, 810, 840) of the app. Electronic devices (101, 200, 300, 400, 500).

5. In any one of paragraphs 1 to 4, The above instructions, when executed by the processor (120), cause the electronic device (101, 200, 300, 400, 500) to: If the first feature information includes information about the second object (615, 715, 745, 815, 845), and the second object (615, 715, 745, 815, 845) is identified as a human shape and takes a specific pose, the target UI object (330, 430, 630, 730, 760, 830, 860, 930, 1110, 1130) is generated based on the first object (220, 625, 725, 755, 825, 855) of the second feature information and the second object (615, 715, 745, 815, 845). Electronic devices (101, 200, 300, 400, 500).

6. In any one of paragraphs 1 to 5, The above target UI objects (330, 430, 630, 730, 760, 830, 860, 930, 1110, 1130) are A target object (335, 635, 735, 765, 835, 865, 935) that is generated by synthesizing a portion of the outer shape of the first object (220, 625, 725, 755, 825, 855) and a portion of the outer shape of the second object (615, 715, 745, 815, 845), and that assumes a pose corresponding to the specific pose, Electronic devices (101, 200, 300, 400, 500).

7. In any one of paragraphs 1 to 6, The above instructions, when executed by the processor (120), cause the electronic device (101, 200, 300, 400, 500) to: If the first feature information includes information about the second object (615, 715, 745, 815, 845) and the second object (615, 715, 745, 815, 845) is not identified as a human shape, the target UI object (330, 430, 630, 730, 760, 830, 860, 930, 1110, 1130) is generated based on the attribute information and / or the second object (615, 715, 745, 815, 845). Electronic devices (101, 200, 300, 400, 500).

8. In any one of paragraphs 1 to 7, The above target UI objects (330, 430, 630, 730, 760, 830, 860, 930, 1110, 1130) are A target object (335, 635, 735, 765, 835, 865, 935) generated based on the above attribute information and a part of the appearance of the first object (220, 625, 725, 755, 825, 855) or including the target object (335, 635, 735, 765, 835, 865, 935) and the second object (615, 715, 745, 815, 845). Electronic devices (101, 200, 300, 400, 500).

9. In any one of paragraphs 1 to 8, The above instructions, when executed by the processor (120), cause the electronic device (101, 200, 300, 400, 500) to: Obtaining auxiliary information (940) for generating the target UI object, and generating the target UI object (330, 430, 630, 730, 760, 830, 860, 930, 1110, 1130) based on at least a part of the first object (220, 625, 725, 755, 825, 855) of the second characteristic information, the first characteristic information, and the auxiliary information (940). Electronic devices (101, 200, 300, 400, 500).

10. In any one of paragraphs 1 to 9, The above auxiliary information (940) is: Information obtained from other apps installed on the electronic device or input from a user of the electronic device (101, 200, 300, 400, 500) through a prompt. Electronic devices (101, 200, 300, 400, 500).

11. In any one of paragraphs 1 to 10, The above instructions, when executed by the processor (120), cause the electronic device (101, 200, 300, 400, 500) to: To share the target UI object (330, 430, 630, 730, 760, 830, 860, 930, 1110, 1130) with one or more electronic devices (101, 200, 300, 400, 500) of a user that are synchronized with the electronic device (101, 200, 300, 400, 500). Electronic devices (101, 200, 300, 400, 500).

12. In electronic devices (101, 200, 300, 400, 500), Memory (130) for storing commands; and Processor (120) executing the above commands Including, The above instructions, when executed by the processor (120), cause the electronic device to: Obtaining at least one first image (320, 420, 521, 523, 620, 720, 750, 820, 850, 920, 1300), generating a second image using at least some of the information and auxiliary information related to the app to generate at least one target UI object (330, 430, 630, 730, 760, 830, 860, 930, 1110, 1130) based on at least some of the features of the at least one first image (320, 420, 521, 523, 620, 720, 750, 820, 850, 920, 1300), and generating at least one target UI object (330, 430, 630, 730, 760, 830, 860, 930, 1110, 1130) associated with the performance of a function of the app based on the second image. to generate 730, 760, 830, 860, 930, 1110, 1130). Electronic devices (101, 200, 300, 400, 500).

13. In paragraph 12, The above instructions, when executed by the processor (120), cause the electronic device to: Generate at least one target UI object using an artificial intelligence model, The above artificial intelligence model, A model learned to generate the at least one target UI object based on at least some of the features of the at least one first image, using information related to an app to generate the at least one target UI object and at least some of the auxiliary information to generate the second image, and generating the at least one target UI object based on the second image. Electronic devices (101, 200, 300, 400, 500).

14. In the operating method of an electronic device (101, 200, 300, 400, 500), Action to acquire images (320, 420, 521, 523, 620, 720, 750, 820, 850, 920, 1300); An action to obtain the first characteristic information of the app that will create the target UI object (330, 430, 630, 730, 760, 830, 860, 930, 1110, 1130); An operation of extracting second feature information from the above images (320, 420, 521, 523, 620, 720, 750, 820, 850, 920, 1300); and An operation including generating the target UI object (330, 430, 630, 730, 760, 830, 860, 930, 1110, 1130) related to the app based on the first feature information and the second feature information. How it works.

15. In paragraph 14, The above second characteristic information is, At least one of the first object, color information used in the image, and the background of the image, How it works.

Images