Electronic device and operation method thereof

Abstract

The present disclosure provides an electronic device. The electronic device may: receive an input for a first image to be set as a wallpaper; recognize at least one object and a background included in the first image; identify an association between the at least one object and at least one widget or icon included in a previous wallpaper; match the at least one widget or icon to the at least one object on the basis of the association; generate a first wallpaper to include the matched at least one widget or icon, by using a generative AI model; display the first wallpaper through a display; and in response to receiving a user confirmation for the first wallpaper, set the first wallpaper as the wallpaper.

Description

Electronic device and its method of operation

[0001] The present disclosure relates to an electronic device and a method of operating the same.

[0002] The default screen of an electronic device is referred to as the wallpaper or home screen, and can be set to an image or animated video according to user settings. Users can customize the wallpaper using fixed icon sets and widget themes. The wallpaper may be selected by the user for aesthetic purposes, while widgets or icons may be added to the wallpaper based on functional needs.

[0003] The information described above may be provided as related art for the purpose of aiding understanding of the present disclosure. No claim or determination is made as to whether any of the foregoing may be applied as prior art related to the present disclosure.

[0004] According to one embodiment, an electronic device may be provided. The electronic device may include at least one processor comprising a display; a memory for storing instructions; and a processing circuit. The instructions may cause the electronic device to perform at least one operation when executed individually or collectively by the at least one processor. The at least one operation may include receiving input for a first image to be set as a background. The at least one operation may include recognizing at least one object and a background included in the first image. The at least one operation may include checking an association between at least one widget or icon included in a previous background and the at least one object. The at least one operation may include matching the at least one widget or icon to the at least one object based on the association. The at least one operation may include generating a first background to include the matched at least one widget or icon using a generative AI model. The at least one operation may include displaying the first background through the display. The above at least one operation may include an operation of setting the first wallpaper as a wallpaper in response to receiving user confirmation for the first wallpaper.

[0005] According to one embodiment, a method of operation of an electronic device may be provided. The method of operation of the electronic device may include at least one operation. The at least one operation may include receiving input for a first image to be set as a wallpaper. The at least one operation may include recognizing at least one object and a background included in the first image. The at least one operation may include checking an association between at least one widget or icon included in a previous wallpaper and the at least one object. The at least one operation may include matching the at least one widget or icon to the at least one object based on the association. The at least one operation may include generating a first wallpaper to include the matched at least one widget or icon using a generative AI model. The at least one operation may include displaying the first wallpaper through the display. The at least one operation may include setting the first wallpaper as a wallpaper in response to receiving user confirmation for the first wallpaper.

[0006] According to one embodiment, a non-transitory storage medium may be provided for storing at least one computer-readable instruction. When executed by at least one processor of an electronic device, the instruction may cause the electronic device to perform at least one operation. The at least one operation may include receiving input for a first image to be set as a background. The at least one operation may include recognizing at least one object and background included in the first image. The at least one operation may include confirming an association between at least one widget or icon included in a previous background and the at least one object. The at least one operation may include matching the at least one widget or icon to the at least one object based on the association. The at least one operation may include generating a first background to include the matched at least one widget or icon based on personal data using a generative AI model. The at least one operation may include displaying the first background through the display. The above at least one operation may include an operation of setting the first wallpaper as a wallpaper in response to receiving user confirmation for the first wallpaper.

[0007] In relation to the description of the drawings, the same or similar reference numerals may be used for identical or similar components.

[0008] FIG. 1 is a block diagram of an electronic device in a network environment according to various embodiments.

[0009] FIG. 2 is an example of a background image automatically generated by an electronic device according to one embodiment of the present disclosure.

[0010] FIG. 3 is a flowchart illustrating the operation of an electronic device according to one embodiment of the present disclosure to automatically generate an image as a background.

[0011] FIG. 4 is an example of an electronic device according to one embodiment of the present disclosure generating a background image including a new widget object.

[0012] FIG. 5 illustrates a block diagram of a generative AI system according to one embodiment of the present disclosure.

[0013] FIG. 6 illustrates a block diagram including functional components of an electronic device according to one embodiment of the present disclosure.

[0014] FIG. 7 is an example of a menu for creating a background image provided by an electronic device according to one embodiment of the present disclosure.

[0015] FIG. 8 is an example of an electronic device according to one embodiment of the present disclosure generating a wallpaper including existing widgets and recommendation widgets.

[0016] FIG. 9 is a flowchart illustrating the operation of an electronic device according to one embodiment of the present disclosure to generate a wallpaper including existing widgets and recommendation widgets.

[0017] FIG. 10 is an example of an electronic device according to one embodiment of the present disclosure displaying a widget within a background screen.

[0018] FIG. 11 is an example of changing the shape of a widget within a background screen using an electronic device according to one embodiment of the present disclosure.

[0019] FIG. 12 is an example of an electronic device according to one embodiment of the present disclosure displaying a plurality of background images.

[0020] The terms used in this disclosure are used merely to describe specific embodiments and are not intended to limit the technical features of this disclosure. For example, a component expressed in the singular form should be understood as a concept including singular or plural components unless the context clearly indicates only the singular form.

[0021] In the present disclosure, each of the phrases such as “A or B,” “at least one of A and B,” “at least one of A or B,” “A, B or C,” “at least one of A, B and C,” and “at least one of A, B, or C” may include any one of the items listed with the corresponding phrase, or all possible combinations thereof. The term “and / or” as used in the present disclosure should be understood to encompass any possible combination by one or more of the plurality of items listed with the corresponding term. Terms such as “first,” “second,” “first,” or “second” as used in the present disclosure may be used merely to distinguish a component from another component and do not limit the components in any other aspect (e.g., importance or order).

[0022] Where it is stated that any (e.g., 1st) component is “coupled,” “connected,” “linked,” “coupled,” “supported,” “connected,” or “contacted” with or without the terms “functionally” or “communicationly,” it includes not only cases where the component is directly coupled, connected, linked, coupled, supported, or contacted with the other component, but also cases where it is indirectly coupled, connected, linked, coupled, supported, or contacted through a third component.

[0023] Terms such as "include" or "have" are intended to specify the existence of the features, numbers, steps, actions, components, parts, or combinations thereof described in this disclosure, and do not preclude the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof. When a component is described as being located "on" another component, this includes not only cases where the component is in contact with the other component, but also cases where another component exists between the two components.

[0024] The expression “configured to” as used in this disclosure may be appropriately substituted, depending on the context, for example, “suitable for,” “capable of,” “designed to,” “modified to,” “made to,” or “capable of.” The term “configured to” does not necessarily mean only that which is “specially designed” in hardware. Instead, in some situations, the expression “device configured to” may mean that the device is “capable of” together with other devices or components. For example, the phrase “device configured (or set) to perform A, B, and C” may mean a device dedicated to performing the said operation, or a general-purpose device capable of performing various operations including said operation.

[0025] Terms used in this disclosure, such as "upper side," "lower side," and "front-rear direction," are defined based on the drawings, and the shape and position of each component are not limited by these terms.

[0026] Although the description in this disclosure is centered on specific embodiments, this disclosure is not limited to such specific embodiments and should be understood to encompass all various modifications, equivalents, and / or substitutions of the various embodiments described in this disclosure. In connection with the description of the drawings, similar reference numerals may be used for similar or related components.

[0027] FIG. 1 is a block diagram of an electronic device in a network environment according to various embodiments.

[0028] Referring to FIG. 1, in a network environment (100), an electronic device (101) may communicate with an electronic device (102) through a first network (198) (e.g., a short-range wireless communication network) or with at least one of an electronic device (104) or a server (108) through a second network (199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (101) may communicate with the electronic device (104) through a server (108). According to one embodiment, the electronic device (101) may include a processor (120), memory (130), input module (150), sound output module (155), display module (160), audio module (170), sensor module (176), interface (177), connection terminal (178), haptic module (179), camera module (180), power management module (188), battery (189), communication module (190), subscriber identification module (196), or antenna module (197). In some embodiments, at least one of these components (e.g., connection terminal (178)) may be omitted from the electronic device (101), or one or more other components may be added. In some embodiments, some of these components (e.g., sensor module (176), camera module (180), or antenna module (197)) may be integrated into a single component (e.g., display module (160)).

[0029] The processor (120) can control at least one other component (e.g., hardware or software component) of the electronic device (101) connected to the processor (120) by executing software (e.g., program (140)), and can perform various data processing or operations. According to one embodiment, as at least part of the data processing or operations, the processor (120) can store commands or data received from other components (e.g., sensor module (176) or communication module (190)) in volatile memory (132), process the commands or data stored in volatile memory (132), and store the resulting data in non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., central processing unit or application processor) or an auxiliary processor (123) that can operate independently or together with it (e.g., graphics processing unit, neural processing unit (NPU), image signal processor, sensor hub processor, or communication processor). For example, if the electronic device (101) includes a main processor (121) and an auxiliary processor (123), the auxiliary processor (123) may be configured to use less power than the main processor (121) or to be specialized for a designated function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as part thereof.

[0030] The auxiliary processor (123) may control at least some of the functions or states associated with at least one component of the electronic device (101) (e.g., display module (160), sensor module (176), or communication module (190)) on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. According to one embodiment, the auxiliary processor (123) (e.g., image signal processor or communication processor) may be implemented as part of another functionally related component (e.g., camera module (180) or communication module (190)). According to one embodiment, the auxiliary processor (123) (e.g., neural network processing unit) may include a hardware structure specialized for processing an artificial intelligence model. The artificial intelligence model may be generated through machine learning. Such learning may be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or through a separate server (e.g., server (108)). The learning algorithm may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model may include a plurality of artificial neural network layers.An artificial neural network may be a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to the hardware structure, the artificial intelligence model may include a software structure, either additionally or substantially.

[0031] The memory (130) can store various data used by at least one component of the electronic device (101) (e.g., processor (120) or sensor module (176)). The data may include, for example, input data or output data for software (e.g., program (140)) and related commands. The memory (130) may include volatile memory (132) or non-volatile memory (134).

[0032] The program (140) may be stored as software in memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0033] The input module (150) can receive commands or data to be used for a component of the electronic device (101) (e.g., processor (120)) from outside the electronic device (101) (e.g., user). The input module (150) may include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0034] The sound output module (155) can output a sound signal to the outside of the electronic device (101). The sound output module (155) may include, for example, a speaker or a receiver. The speaker may be used for general purposes, such as multimedia playback or recording playback. The receiver may be used to receive incoming calls. According to one embodiment, the receiver may be implemented separately from the speaker or as part thereof.

[0035] The display module (160) can visually provide information to an external (e.g., user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling said device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of the force generated by said touch.

[0036] The audio module (170) can convert sound into an electrical signal or, conversely, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150) or output sound through the sound output module (155) or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphones) connected directly or wirelessly to the electronic device (101).

[0037] The sensor module (176) can detect the operating state of the electronic device (101) (e.g., power or temperature) or the external environmental state (e.g., user state) and generate an electrical signal or data value corresponding to the detected state. According to one embodiment, the sensor module (176) may include, for example, a gesture sensor, a gyroscope sensor, a barometric pressure sensor, a magnetic sensor, an accelerometer sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biosensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0038] The interface (177) may support one or more specified protocols that can be used for the electronic device (101) to be connected directly or wirelessly to an external electronic device (e.g., electronic device (102)). According to one embodiment, the interface (177) may include, for example, a high definition multi-media interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0039] The connection terminal (178) may include a connector through which the electronic device (101) can be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0040] The haptic module (179) can convert an electrical signal into a mechanical stimulus (e.g., vibration or movement) or an electrical stimulus that the user can perceive through tactile or kinesthetic senses. According to one embodiment, the haptic module (179) may include, for example, a motor, a piezoelectric element, or an electric stimulation device.

[0041] The camera module (180) can capture still images and video. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0042] The power management module (188) can manage the power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented, for example, as at least part of a power management integrated circuit (PMIC).

[0043] The battery (189) can supply power to at least one component of the electronic device (101). According to one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0044] The communication module (190) can support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between an electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may include one or more communication processors that operate independently of the processor (120) (e.g., application processor) and support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., cellular communication module, short-range wireless communication module, or GNSS (global navigation satellite system) communication module) or a wired communication module (194) (e.g., LAN (local area network) communication module, or power line communication module). The corresponding communication module among these communication modules can communicate with an external electronic device (104) through a first network (198) (e.g., a short-range communication network such as Bluetooth, WiFi (wireless fidelity) direct, or IrDA (infrared data association)) or a second network (199) (e.g., a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules may be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can identify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) using subscriber information (e.g., International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module (196).

[0045] The wireless communication module (192) can support 5G networks and next-generation communication technologies following 4G networks, for example, new radio access technology. NR access technology can support high-speed transmission of high-capacity data (enhanced mobile broadband (eMBB)), minimization of terminal power and connection of multiple terminals (massive machine type communications (mMTC)), or high reliability and low latency (ultra-reliable and low-latency communications (URLLC)). The wireless communication module (192) can support a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate, for example. The wireless communication module (192) can support various technologies for securing performance in the high-frequency band, such as beamforming, massive MIMO (multiple-input and multiple-output), full-dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or largescale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), external electronic device (e.g., electronic device (104)), or network system (e.g., second network (199)). According to one embodiment, the wireless communication module (192) can support a Peak data rate (e.g., 20 Gbps or more) for realizing eMBB, loss coverage (e.g., 164 dB or less) for realizing mMTC, or U-plane latency (e.g., downlink (DL) and uplink (UL) each 0.5 ms or less, or round trip 1 ms or less) for realizing URLLC.

[0046] An antenna module (197) can transmit a signal or power to or from an external source (e.g., an external electronic device). According to one embodiment, the antenna module (197) may include an antenna comprising a radiator made of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). According to one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as a first network (198) or a second network (199), may be selected from the plurality of antennas, for example, by a communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device through the selected at least one antenna. According to some embodiments, in addition to the radiator, other components (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as part of the antenna module (197).

[0047] According to various embodiments, the antenna module (197) may form a mmWave antenna module. According to one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent to a first surface (e.g., bottom surface) of the printed circuit board and capable of supporting a specified high frequency band (e.g., mmWave band), and a plurality of antennas (e.g., array antennas) disposed on or adjacent to a second surface (e.g., top surface or side surface) of the printed circuit board and capable of transmitting or receiving a signal of the specified high frequency band.

[0048] At least some of the above components can be connected to each other via a communication method between peripheral devices (e.g., bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)) and exchange signals (e.g., commands or data) with each other.

[0049] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) through a server (108) connected to a second network (199). Each of the external electronic devices (102, or 104) may be the same or different type of device as the electronic device (101). According to one embodiment, all or part of the operations performed on the electronic device (101) may be performed on one or more of the external electronic devices (102, 104, or 108). For example, if the electronic device (101) needs to perform a function or service automatically or in response to a request from a user or another device, the electronic device (101) may request one or more external electronic devices to perform at least part of the function or service instead of performing the function or service itself or additionally. One or more external electronic devices that receive the above request may execute at least part of the requested function or service, or additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may provide the result as is or additionally processed as at least part of the response to the request. For this purpose, for example, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used. The electronic device (101) may provide ultra-low latency services using, for example, distributed computing or mobile edge computing. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server using machine learning and / or neural networks. According to one embodiment, the external electronic device (104) or the server (108) may be included within the second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0050] The electronic device according to the various embodiments disclosed in this document may be of various forms. The electronic device may include, for example, a portable communication device (e.g., a smartphone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a consumer electronics device. The electronic device according to the embodiments of this document is not limited to the devices described above.

[0051] FIG. 2 is an example of a background image automatically generated by an electronic device according to one embodiment of the present disclosure.

[0052] An electronic device according to one embodiment (e.g., the electronic device (101) of FIG. 1) can analyze an image to be set as a background and generate a new background by matching widgets and / or icons included in the existing background to the image. When a user selects an image to be used as a background in order to change the background, the device can analyze the image, such as by analyzing the style of the image, object identification, or object characteristics / region analysis, to determine a widget or icon that can be associated with the identified object, and recommend to the user that the object included in the image be used as the determined widget or icon. For example, it can recommend an object corresponding to a widget or icon in the previous background and additionally recommend a widget or icon that the user might use.

[0053] The first screen (210) displays an image selected to be set as a background by user input. The first screen (210) may be a picture of a desk containing office supplies. The electronic device (101) can identify various objects on the desk (e.g., a calendar (211), a flowerpot (212), a clock (213), a picture (214, 215), a notebook, or a book). The electronic device (101) can extract contextual information, such as desk interior, organization, or studying, from the image of the first screen (210).

[0054] According to one embodiment, the second screen (220) displays a background image created by recommending an object included in the first screen (210) as a widget. For example, the electronic device (101) may recommend a calendar (211) as a calendar widget (221), a clock (213) as a clock widget (222), or a photo (215) as a gallery widget (223) in the first screen (210). The second screen (220) may include a background image creation button (generate) (225) that allows user confirmation of the automatically generated background image to be entered. Upon receiving user input for the background image creation button (225), the electronic device (101) may set and save the second screen (220) as a background image.

[0055] An electronic device (101) according to one embodiment can analyze an image set as a wallpaper using AI, identify the attributes of the image and objects included in the image, and recommend a widget. The electronic device (101) can analyze the association between an existing widget and an object in the image to link a suitable widget to the wallpaper. By utilizing the user's personal data to recommend a widget with a high probability of use, the electronic device (101) can expect a high level of personalization even when the same image is set as a wallpaper by different users. The electronic device (101) can create a wallpaper by changing the widgets and / or icons included in the previous wallpaper into a widget shape suitable for the new image.

[0056] FIG. 3 is a flowchart illustrating the operation of an electronic device according to one embodiment of the present disclosure to automatically generate an image as a background.

[0057] An electronic device (101) according to one embodiment (e.g., the electronic device (101) of FIG. 1) can generate a wallpaper including existing widgets and icons using an object included in a first image to be set as a wallpaper. In the following embodiments, each operation may be performed sequentially, but is not necessarily performed sequentially. For example, the order of each operation may be changed, and at least two operations may be performed in parallel.

[0058] In operation 310, an electronic device (101) according to one embodiment may receive a first image to be set as a wallpaper. For example, the user may select a menu to set an image stored in a gallery as a wallpaper, or select a new image from a wallpaper creation menu.

[0059] In operation 320, an electronic device (101) according to one embodiment may recognize at least one object and / or background included in a first image. For example, the electronic device (101) may analyze the first image to identify attributes of the image and objects included in the image. Attributes of the image may include color, material, style, or mood, etc. The electronic device (101) may identify objects (e.g., trees) or people (e.g., women), identify people related to the user (e.g., friends), and may also identify specific identifications. Specific identifications may be, for example, a person holding an umbrella or a happy child.

[0060] In operation 330, an electronic device (101) according to one embodiment can determine the association between a widget or icon of a previous wallpaper and an object of a first image. For example, the electronic device (101) can use an AI model to quantitatively determine the association by comparing the attributes of the object analyzed in the first image with the attributes of the existing widget, if there is a widget included in the previous wallpaper. The AI ​​model can calculate the association by analyzing the object in the input image and comparing it with the attributes of the existing widget. The AI ​​model analyzes attributes such as the color, shape, or location of the image and can understand the context of the object. The meaning of the analyzed object can be compared with the widget's function, visual style, and frequency of use. The AI ​​model can score the similarity between the object and the widget and recommend a suitable widget. For example, it can recommend matching an existing weather widget with a "cloud object" of the new wallpaper.

[0061] In operation 340, an electronic device (101) according to one embodiment may match a widget or icon to an object of a first image based on association. For example, if the electronic device (101) determines that the association with an existing widget is higher than a threshold level, it may match and place the existing widget on the object. If there are multiple widgets with high association, the electronic device (101) may recommend them in order of high association.

[0062] In operation 350, an electronic device (101) according to one embodiment may generate a first wallpaper that includes a widget or icon matched based on personal data using a generative AI model. For example, the electronic device (101) may generate a prompt to naturally change a first image into a first wallpaper that includes a widget / icon matched based on personal data, and input this into a generative AI model to obtain the first wallpaper. For example, personal data may include the user's preferences or usage history. The matched widget / icon may be arranged on the first wallpaper to have visual coherence. The electronic device (101) may optimize an object corresponding to the matched widget / icon based on the user's personal data.

[0063] In operation 360, an electronic device (101) according to one embodiment may display a first background image in a preview form through a display. For example, the electronic device (101) may further display widget information (e.g., name) in the preview form so that objects and widgets included in the first background image are distinguished from each other. For example, the electronic device (101) may provide an editing function for the first background image. For example, it may receive user input to change the position, size, or shape of a first widget included in the first background image and regenerate the first background image by reflecting this input. In response to moving the position of the first widget, the electronic device (101) may regenerate the first background image by automatically changing at least one of the color, size, or shape of the first widget to reflect the position after moving. The electronic device (101) may output the first background image through a display as it is regenerated by user editing.

[0064] In operation 370, an electronic device (101) according to one embodiment may set a first wallpaper upon receiving user confirmation input for the first wallpaper. If the electronic device (101) does not receive user confirmation for the first wallpaper, it may delete the first wallpaper. When the electronic device (101) saves the first wallpaper, it may store and manage it as user preference data. When the electronic device (101) cancels the first wallpaper, it may reflect this in the user preference data.

[0065] An electronic device according to one embodiment of the present disclosure (e.g., the electronic device (101) of FIG. 1) comprises a display (e.g., the display module (160) of FIG. 1); a memory (e.g., the memory (130) of FIG. 1) for storing instructions; and at least one processor (e.g., the processor (120) of FIG. 1) comprising a processing circuit; wherein the instructions, when executed individually or collectively by the at least one processor (120), may cause the electronic device (101) to perform at least one operation. According to one embodiment, the at least one operation may include receiving input for a first image to be set as a background image and recognizing at least one object and background included in the first image. The at least one operation may include confirming an association between at least one widget or icon included in the previous background image and the at least one object. The at least one operation may include matching the at least one widget or icon to the at least one object based on the association. At least one operation may include an operation to generate a first wallpaper including at least one matched widget or icon using a generative AI model. At least one operation may include an operation to display the first wallpaper through the display (160). At least one operation may include an operation to set the first wallpaper as a wallpaper in response to receiving user confirmation for the first wallpaper.

[0066] According to one embodiment, the electronic device (101) may match the first widget or icon to the first object in response to a determination that the first widget or icon among at least one widget or icon included in the previous background image has a high correlation of at least a threshold level or higher with the first object among at least one object included in the first image, and may create a second object and match the second widget or icon in response to a determination that there is no object among at least one widget or icon among at least one widget or icon included in the previous background image and at least one object included in the first image has a high correlation of at least a threshold level or higher.

[0067] According to one embodiment, the electronic device (101) requests the generative AI model to generate the first wallpaper based on personal data, and the personal data may include at least one of a user's profile, age, gender, language, social data, preferences, interests, event information, schedule information, location data, activity pattern information, activity data, usage history, search history, or gallery data.

[0068] According to one embodiment, the electronic device (101) identifies a first widget among a plurality of applications installed on the electronic device (101) that is determined to have a higher probability of being used by a user based on personal data than a threshold level, and can match the first widget to a third object among the at least one object included in the first image that is determined to be associated with the first widget.

[0069] According to one embodiment, an electronic device (101) receives user input regarding at least one widget or icon included in the first background image and can change the position, size, shape, color, or name of the at least one widget or icon according to the user input using the generative AI model.

[0070] According to one embodiment, the electronic device (101) may place at least one widget or icon included in the first background on the first background by reflecting at least one of the position or size in the previous background.

[0071] According to one embodiment, an electronic device (101) can acquire real-time data related to at least one widget included in the first background image, update the data of the at least one widget according to the real-time data, regenerate the first background image reflecting the update data of the at least one widget while maintaining visual consistency using the generative AI model, and display the first background image.

[0072] According to one embodiment, the electronic device (101) can display at least one object corresponding to at least one widget or icon included in the first background image, distinguished from the object included in the first background image.

[0073] According to one embodiment, the electronic device (101) may display at least one of the name, shadow, or border of the at least one widget or icon.

[0074] A method of an electronic device according to one embodiment of the present disclosure may include: receiving input for a first image to be set as a wallpaper; recognizing at least one object and a background included in the first image; confirming an association between at least one widget or icon included in a previous wallpaper and at least one object; matching at least one widget or icon to at least one object based on the association; generating a first wallpaper to include at least one matching widget or icon using a generative AI model; displaying the first wallpaper through a display; and setting the first wallpaper as a wallpaper in response to receiving user confirmation for the first wallpaper.

[0075] According to one embodiment, the operation of matching the at least one widget or icon to the at least one object based on the association in the method may include: the operation of matching the first widget or icon to the first object in response to a determination that the first widget or icon among the at least one widget or icon included in the previous background image has a high association of at least a threshold level or higher with the first object among the at least one object included in the first image; and the operation of creating a second object and matching the second widget or icon in response to a determination that there is no object among the at least one widget or icon among the at least one widget or icon included in the previous background image and at least one object included in the first image has a high association of at least a threshold level or higher.

[0076] According to one embodiment, the operation of generating a first wallpaper including at least one matched widget or icon using the generative AI model in the method comprises requesting the generative AI model to generate the first wallpaper based on personal data, wherein the personal data may include at least one of a user's profile, age, gender, language, social data, preferences, interests, event information, schedule information, location data, activity pattern information, activity data, usage history, search history, or gallery data.

[0077] According to one embodiment, the method may further include: identifying a first widget among a plurality of applications installed on the electronic device that is determined to have a higher probability of being used by a user than a threshold level based on personal data; and matching the first widget to a third object among the at least one object included in the first image that is determined to be associated with the first widget.

[0078] According to one embodiment, the method may further include: receiving user input for at least one widget or icon included in the first background; and changing the position, size, shape, color, or name of the at least one widget or icon according to the user input using the generative AI model.

[0079] According to one embodiment, the method may further include the operation of placing at least one widget or icon included in the first background on the first background by reflecting at least one of the position or size in the previous background.

[0080] According to one embodiment, the method may further include: acquiring real-time data related to at least one widget included in the first background; updating data of the at least one widget according to the real-time data; regenerating the first background that reflects the update data of the at least one widget while maintaining visual consistency using the generative AI model; and displaying the first background.

[0081] According to one embodiment, the method may further include an operation to display at least one object corresponding to at least one widget or icon included in the first background so as to be distinguished from the object included in the first background.

[0082] According to one embodiment, in the method, at least one of the name, shadow, or border of the at least one widget or icon may be displayed.

[0083] According to one embodiment of the present disclosure, a non-transitory storage medium may be provided for storing at least one computer-readable instruction. The instruction may perform at least one operation when executed by at least one processor of an electronic device. According to one embodiment, at least one operation may include receiving input for a first image to be set as a wallpaper. At least one operation may include recognizing at least one object and background included in the first image. At least one operation may include confirming an association between at least one widget or icon included in a previous wallpaper and at least one object. At least one operation may include matching at least one widget or icon to at least one object based on the association. At least one operation may include generating a first wallpaper to include the matched at least one widget or icon using a generative AI model. At least one operation may include displaying the first wallpaper through a display. At least one operation may include setting the first wallpaper as a wallpaper in response to receiving user confirmation for the first wallpaper.

[0084] According to one embodiment, in the storage medium, the operation of matching the at least one widget or icon to the at least one object based on the association may include: the operation of matching the first widget or icon to the first object in response to a determination that the first widget or icon among the at least one widget or icon included in the previous background image and the first object among the at least one object included in the first image have a high association of at least a threshold level; and the operation of creating a second object and matching the second widget or icon in response to a determination that there is no object among the at least one widget or icon among the at least one widget or icon included in the previous background image and the second widget or icon having a high association of at least a threshold level.

[0085] FIG. 4 is an example of an electronic device according to one embodiment of the present disclosure generating a background image including a new widget object.

[0086] An electronic device according to one embodiment (e.g., the electronic device (101) of FIG. 1) may include widgets or icons included in a previous wallpaper as new widget objects to match the new wallpaper. When a user selects an image to use as a wallpaper to change the wallpaper, the electronic device (101) may determine whether there are widgets or icons in the previously set wallpaper. The electronic device (101) may analyze the image, such as the style of the image for change, object identification, or object characteristic / region analysis, and based on the analysis results, regenerate the widgets or icons included in the previous wallpaper into widgets or icons associated with the wallpaper through generative AI and display them on the changed wallpaper. For example, the electronic device (101) may regenerate weather widgets, phone, or camera app icons in a form that matches the wallpaper of the third screen (430) being changed, such as the first screen (410).

[0087] The first screen (410) is the background screen of the current electronic device (101). The first screen (410) includes a weather widget (412), a phone app (413), or a camera app (414) with a flower picture as the background (411). On the first screen (410), a background screen creation menu can be executed.

[0088] The second screen (420) is an example of a background image creation menu that lists and displays proposed background images (421) and can display a gallery (422) in conjunction with the user so that the user can directly select a new image. The electronic device (101) can receive user input (402) for selecting the first image (4211) among the proposed background images (4211, 4212, 4213).

[0089] The third screen (430) is a new wallpaper (431) that includes new widget objects (432, 433, 434) by reflecting the weather widget (412), phone app (413), and camera app (414) in the first image (4211). The electronic device (101) can match a widget or icon to an object if there is an object with a correlation above a threshold level among at least one object included in the first image (4211) among the widgets and icons included in the first screen (410). If there is no object with a correlation above a threshold level among at least one object included in the first image (4211) among the widgets and icons included in the first screen (410), the electronic device (101) can create a new widget / icon object and add it to the first image (4211).

[0090] The first image (4211) is an image containing a view seen through a window in front of a desk, and the previous weather widget (412), phone app (413), and / or camera app (414) do not match the first image (4211). The electronic device (101) can generate a new wallpaper (431) by using a generative AI model to include a cloud-shaped weather widget (432), an old-fashioned phone-shaped phone app (433), or a digital camera-shaped camera app (434) in the first image (4211). The electronic device (101) can generate the new wallpaper (431) by reflecting at least one of the position and size of the widget (412) and icons (413, 414) included in the first screen (410). The phone app (433) can be placed in the lower left of the new wallpaper (431) by reflecting that it was placed in the lower left of the first screen (410), which is the existing wallpaper. The phone app (433) can be created in the shape of an old-fashioned telephone in a style that matches other objects (e.g., a coffee cup, or a book) on the table in the bottom left. The camera app (434) can be placed in the bottom right of the new wallpaper (431), reflecting that it is placed in the bottom right of the first screen (410), which is the existing wallpaper. The camera app (434) can be created in the shape of a digital camera in a style that matches other objects (e.g., a coffee cup, or a book) on the table in the bottom left. The phone app (433) and the camera app (434) can be created in a size that matches the size of surrounding objects (e.g., a table, a book, or a coffee cup).

[0091] The electronic device (101) can provide an editing function for a new wallpaper (431). The electronic device (101) receives user input regarding a widget (432) or an icon (433, 434) included in the wallpaper (431), and can change the position, size, shape, color, or name of the widget or icon according to the user input using a generative AI model. The weather widget (432) of the third screen (430) may have its widget data changed according to real-time data. The electronic device (101) receives real-time data for the weather widget (432) from a server and can update the data of the weather widget (432) according to the real-time data. The electronic device (101) can regenerate and output a new wallpaper by reflecting the weather widget update data while maintaining visual consistency using a generative AI model. The electronic device (101) can dynamically update the widget data and the wallpaper together. The electronic device (101) can automatically update the wallpaper by using a generative AI model to immediately reflect changes in the data of a configured widget (e.g., weather, or time). The electronic device (101) can provide a continuously changing appearance by dynamically updating the widget generated according to the user's personal data and real-time data changes, rather than simply placing a static widget. Through this, the electronic device (101) can provide a living user experience where the widget and the wallpaper are integrated. For example, if the weather widget changes from "clear" to "rain," the clear sky in the area containing the weather widget object on the wallpaper can be converted into a rainy sky. The electronic device (101) can provide visual consistency by maintaining a dynamic and lively home screen that interacts with the wallpaper whenever the widget is updated.

[0092] The third screen (430) may include a wallpaper creation button (435) for receiving user confirmation for an automatically generated wallpaper. Upon receiving user input for the wallpaper creation button (435), the electronic device (101) may set and save the wallpaper (431) as a new wallpaper of the electronic device (101).

[0093] FIG. 5 illustrates a block diagram of a generative AI system according to one embodiment of the present disclosure.

[0094] An electronic device according to one embodiment (e.g., the electronic device (101) of FIG. 1) can generate a background using a generative AI model. The generative AI system may include a user query / response interface (510), an AI framework (520), a knowledge repository (530), an application / service component (540), and a generative AI model (550).

[0095] The user query / response interface (510) can receive a query as user input and output a corresponding response. User input may include, for example, natural language, images, voice, video, documents, or context data. Context data may include various additional information at the time when user input occurs. For example, information about the application currently being used by the user and the user's location information may be context data. User input may be in the form of a mixture of various data. Through the user query / response interface (510), user input including at least one of user voice (natural language), sound, images, video, key button input, touch input, gesture input, location information, or health sensor data may be received. The user query / response interface (510) may also receive user input through other devices (e.g., wearable devices) that are connected to the electronic device (101) in communication, in addition to the input device of the electronic device (101) (e.g., keyboard, touchpad, keyboard, mouse, microphone, or sensor).

[0096] The user query / response interface (510) can output an answer obtained based on a generative AI model. The answer may be provided in the form of natural language or specific content, and may also be provided in the form of an action requested by the user.

[0097] The AI ​​framework (520) can receive user input and coordinate and control each component necessary to perform the user's intent based on the user's query. The AI ​​framework (520) may include a prompt design component (521), an API / Plug-in management component (522), and a refiner component (523).

[0098] User input received from the user query / response interface (510) can be transmitted to a prompt design component (521). The prompt design component (521) can be used to generate prompts suitable for inputting user input into a large language model (LLM) or a large multi-modal model (LMM). The prompt design component (521) may be an AI component that uses machine learning algorithms or neural networks to develop better prompts over time. Based on user input, the prompt design component (521) can generate prompts by accessing a knowledge repository (530) containing user preference data, prompt libraries, and prompt examples, and can transmit the generated prompts to the LLM or LMM.

[0099] The API / Plug-in management component (522) can perform the role of communicating with external information when there is a request for additional information when passing user input as input to a generative model. The API / Plug-in management component (522) establishes a channel to communicate with the outside of the AI ​​Interface via the API, and can enable access to various data sources (e.g., knowledge repository (530)) through the established channel. Additionally, the API / Plug-in management component (522) can request the application / service component (540) via the API to perform an action that ultimately executes the user input, rather than an intermediate result, in the application or service. The information obtained from the outside may be used to generate a prompt in the prompt design component (521) along with the user input, or it may be passed as input to the generative model.

[0100] The refinement component (523) can fine-tune the output from the generative model. For example, the refinement component (523) can verify whether the content generated through LLM and / or LMM is irrelevant, contains biased content, or contains harmful content. Additionally, the refinement component (523) can determine the extent to which the output matches what the user wants and, if additional processing is required, proceed with that process. Furthermore, the refinement component (523) can configure and provide hints to the user to avoid unwanted output.

[0101] A generative AI model (550) can generally refer to an artificial intelligence neural network that generates new forms of data based on user input information. A generative AI model (550) may include a model that generates images and / or a model that generates language. Models that generate images include, but are not limited to, GANs (generative adversarial networks) and VAEs (variational auto encoders), and examples include Diffusion-based generative models that use VAEs and Transformer structures. Models that generate language are models trained to output the most statistically appropriate output value based on input values, and examples include models such as CHAT-GPT 3 and CHAT-GPT 4. There are also LMMs (large multimodal models) that can recognize various forms of data input, such as text, images, and voice, and generate new data corresponding to them.

[0102] FIG. 6 illustrates a block diagram including functional components of an electronic device according to one embodiment of the present disclosure.

[0103] An electronic device according to one embodiment (e.g., the electronic device (101) of FIG. 1) may include one or more functional components for automatically generating a background for user drawing input. The electronic device (101) may include an input processing module (610), personal data (620), a generative AI model (630) and / or a display (640). The input processing module (610) may include an image analysis module (611), a widget / icon analysis module (612), a widget recommendation module (613), and a prompt assistant manager (615).

[0104] An electronic device (101) according to one embodiment may receive a background image selection (601). For example, the electronic device (101) may receive user input to set an image selected from a background creation menu or gallery as a background.

[0105] According to one embodiment, an image analysis module (611) can analyze the attributes of an image and identify objects included in the image. Image attributes may include, for example, image color, style, texture, information about objects within the image, characteristics of objects, or location areas of objects. The image analysis module (611) may include an AI model or an image analysis learning model, etc. For example, the image analysis module (611) can use an AI model to identify "sky, trees, or clouds" as objects included in a landscape photograph with warm tones. The AI ​​model can analyze the input image to extract attributes such as color, style, or texture. The AI ​​model can learn the association between the subject of the image and text by utilizing a context encoder (e.g., CLIP, ELIP). Contrastive learning (CLIP) can connect the image and text to detect specific elements (e.g., clouds) and extend them into contextual information such as the sky. BLIP (bootstrapping language-image pretraining) can recognize detailed objects in an image (e.g., a person holding an umbrella) with greater precision. A style encoder can extract style features and texture information to be used for background design. The electronic device (101) can understand the overall mood and visual characteristics of the image through image analysis.

[0106] According to one embodiment, the image analysis module (611) can identify major objects included in an image and determine the category of said objects (e.g., clock, calendar, or weather). The AI ​​model can classify objects within the image and identify their locations and boundaries. The image analysis module (611) can identify objects such as "calendar" or "cloud" and extract their locations by separating objects at the pixel level using semantic segmentation technology. Based on the meaning of the objects detected by the content encoder, it can recommend a suitable widget for each object (e.g., a schedule widget is appropriate for a calendar object) or place the widget in an appropriate location within the background. Through this, the electronic device (101) can create a user-customized UI by combining the visual information and purpose of use of the objects.

[0107] According to one embodiment, the widget / icon analysis module (612) can analyze the association between a widget in an existing wallpaper and an object included in a new wallpaper. Based on the association, the widget / icon analysis module (612) can match an existing widget to an object in the new wallpaper.

[0108] According to one embodiment, the widget recommendation module (613) can recommend a suitable widget or icon based on an identified object. For example, the widget recommendation module (613) can quantitatively determine the correlation by comparing the category of a widget or icon provided by at least one application installed in the electronic device (101) with the category of the identified object. If the widget recommendation module (613) determines that the category of the identified object and the category of the widget / icon have a correlation higher than a threshold level, it can match (recommend) a widget / icon that is highly correlated with the object. For example, the widget recommendation module (613) can recommend a widget based on the shape, location, or size of the object. For example, the electronic device (101) can recommend a suitable widget or icon, such as a notepad, gallery, or calendar, corresponding to a square-shaped object. The widget recommendation module (613) can recommend a new widget based on personal data. The widget recommendation module (613) can create and recommend suitable widgets based on profile, preference, or activity data analysis results.

[0109] According to one embodiment, the widget recommendation module (613) may obtain input regarding user selection or confirmation of a recommended widget or icon. The widget recommendation module (613) may set and save the recommended widget selected by the user. The widget recommendation module (613) may delete recommended widgets that are not selected by the user. The widget recommendation module (613) may save and manage the user selection as a user preference in personal data.

[0110] According to one embodiment, the prompt assistant manager (615) interacts with the image analysis module (611), the widget / icon analysis module (612), and the widget recommendation module (613) to generate a prompt for the generative AI model (630), and may reflect personal data (620). The electronic device (101) can automatically generate a wallpaper containing widgets / icons from a simple image using the generative AI model (630). The electronic device (101) can analyze the input image and generate a prompt for generating a wallpaper based on personal data.

[0111] According to one embodiment, the personal data (620) may include app data, usage data, or system data as raw data (621). The raw data (621) may be stored in the personal data (620) as is, or information processed through the data platform (622) may be stored in the personal data (620). The raw data (621) may be transmitted to the input processing module (610) upon the request of the input processing module (610).

[0112] The prompt may request the generative AI model (630) to generate a wallpaper. The electronic device (101) may reflect the user's personal preferences by reflecting personal data in the prompt. The electronic device (101) may store and manage personal data (620) based on the user's profile (e.g., relationships), preferences (e.g., intimacy, events, or preferred places), context (e.g., places, behaviors, or patterns), or activities (e.g., travel, or exercise). The electronic device (101) may utilize an AI analysis model (e.g., LLM, or LMM) to extract information from the acquired raw data (621). The electronic device (101) may determine which widgets the user is likely to use based on the user's personal data. The personal data may include at least one of the user's profile, age, gender, language, social data, preferences, interests, event information, schedule information, location data, activity pattern information, activity data, usage history, search history, or gallery data.

[0113] According to one embodiment, the generative AI model (630) can generate a new wallpaper by changing the widgets and / or icons of the previous wallpaper to match the new wallpaper. The generative AI model (630) may include a model that generates images and / or a model that generates language, and may include at least a portion of the generative AI model (550) of FIG. 5. The generative AI model (630) may be included in an electronic device (101) or in an external device (e.g., an external server). In the case of an external device, the generative AI model (630) may be connected to the electronic device (101) to transmit input values ​​including prompts and receive the wallpaper.

[0114] According to one embodiment, the display (640) can output an automatically generated wallpaper. The characteristics of the display (640) (e.g., screen size, resolution, or aspect ratio) may affect the size of the automatically generated wallpaper and the placement of objects included in the wallpaper. The display (640) can regenerate and display a wallpaper that updates the widget by reflecting real-time data included in the widget.

[0115] FIG. 7 is an example of a menu for creating a background image provided by an electronic device according to one embodiment of the present disclosure.

[0116] An electronic device according to one embodiment (e.g., the electronic device (101) of FIG. 1) may provide a menu for creating a background image.

[0117] According to one embodiment, the first wallpaper generation screen (710) may include a menu (711) for selecting a wallpaper style, a wallpaper change menu (712), or an automatic wallpaper generation menu (713). When the electronic device (101) receives user input (701) for the automatic wallpaper generation menu (713), an image selection screen (720) for automatic wallpaper generation may be displayed. The image selection screen (720) may include a portion (721) for displaying recommended background images based on personal data and a portion (722) linked with gallery images that can be directly selected by the user. The electronic device (101) may receive user input (702) for a first background image among the recommended background images. The electronic device (101) may generate the first background image as a new wallpaper including existing widgets / icons or recommended widgets / icons. The electronic device (101) can determine which widgets a user is likely to use based on personal data (e.g., profile, preference, situation, or activity) identified through AI, and can recommend objects within the first background image identified by analyzing the first background image.

[0118] FIG. 8 is an example of an electronic device according to one embodiment of the present disclosure generating a wallpaper including existing widgets and recommendation widgets.

[0119] An electronic device according to one embodiment (e.g., the electronic device (101) of FIG. 1) can generate a wallpaper that includes existing widgets included in a previous wallpaper and new widgets recommended based on user personal data.

[0120] The first image (810) is a previous background image and may include a clock icon (8111), a calendar icon (8112), or a gallery icon (8113) on a flower-shaped background. According to one embodiment, the electronic device (101) may analyze the attributes of the second image (820) and identify objects included in the second image (820) upon receiving user input to create the second image (820) as a background image. The second image (820) may include several office supplies (e.g., a vase, a calendar, a photograph, a book, a clock, or a picture frame) placed on a desk. The electronic device (101) may change the calendar among the objects included in the second image (820) into a widget (8212) of a calendar application corresponding to the calendar icon (8112) of the previous background image (810) (812 operation). The electronic device (101) can change the clock among the objects included in the second image (820) to the clock icon (8111) of the previous wallpaper (810) to the clock application widget (8211) (811 operation). The electronic device (101) can change the photo among the objects included in the second image (820) to the gallery icon (8113) of the previous wallpaper (810) to the gallery application icon (8213) (813 operation). The electronic device (101) can reflect at least one of the position or size of the widget or icon included in the existing wallpaper (810) in the new wallpaper (820). For example, reflecting that the clock icon (8111) was placed on the left and the gallery icon (8113) was placed on the right in the existing wallpaper (810), the clock widget (8211) can be placed on the left and the gallery icon (8213) on the right in the new wallpaper (820).

[0121] According to one embodiment, the electronic device (101) can identify a first widget among a plurality of applications installed on the electronic device (101) that is determined to have a higher probability of being used by the user than a threshold level based on personal data. The electronic device (101) can determine a third object (e.g., a photo) among at least one object included in a second image (820) designated as a new background that is determined to be associated with the first widget with a high probability of being used. For example, the electronic device (101) can recommend an application icon (8214) for calling a spouse based on the user's personal data and the activity pattern of calling a spouse. The electronic device (101) can recommend a direct call icon (8214) for a person with high usage frequency among the shortcut settings. The electronic device (101) can identify a spouse photo among the objects included in the second image (820) and recommend a direct call icon (8214) for a spouse designated in the shortcut settings of the phone application by matching it to the identified photo. The electronic device (101) can recommend a direct call icon (8214) to the spouse using the profile picture included in the spouse information included in the contacts.

[0122] FIG. 9 is a flowchart illustrating the operation of an electronic device according to one embodiment of the present disclosure to generate a wallpaper including existing widgets and recommendation widgets.

[0123] An electronic device (101) according to one embodiment (e.g., the electronic device (101) of FIG. 1) can generate a wallpaper including existing widgets and recommendation widgets using an object included in a first image to be set as a wallpaper. The electronic device (101) can generate a wallpaper including existing icons and recommendation icons. In the description of FIG. 9, widgets are described as examples. In the following embodiments, each operation may be performed sequentially, but is not necessarily performed sequentially. For example, the order of each operation may be changed, and at least two operations may be performed in parallel.

[0124] In operation 910, an electronic device (101) according to one embodiment can identify an object included in a first image. The first image may be one selected by a user to set as a new wallpaper. The electronic device (101) can identify the included object by analyzing the first image using an image analysis module (e.g., an image analysis AI model).

[0125] In operation 920, an electronic device (101) according to one embodiment may identify a first widget with high availability based on personal data. The personal data may include, for example, a profile, user characteristics (e.g., age, gender, or language), friend relationships (e.g., contacts, or message history), social data (e.g., social network connection information, or usage history), preferences (e.g., user-set interests, favorites list, or frequently used apps and services), interests, events, schedule information (e.g., schedules saved in a calendar, notification settings, or scheduled texts), location and activity patterns (e.g., location information, travel routes, or regularity of places visited), activity data (e.g., activity records for exercise, travel, or shopping), usage history of the electronic device (101) (e.g., frequency of use of a specific app, or screen usage time), search history (e.g., keywords searched by the user, or topics of interest), or gallery data (e.g., images and videos in the gallery). The electronic device (101) can identify widgets with a high likelihood of use among the widgets included in the electronic device (101) based on personal data analysis. For example, if the user frequently searches for nature or travel-related content, the electronic device (101) can recommend widgets related to travel, maps, and schedules based on drawings. The electronic device (101) can generate images and objects suitable for travel to place the recommended widgets on the background.

[0126] In operation 930, an electronic device (101) according to one embodiment may select a first object that is associated with a first widget among the objects of a first image. For example, the electronic device (101) may check the association between the category of objects of the first image and the category of the first widget and select a first object in which the association is higher than a threshold level.

[0127] In operation 940, an electronic device (101) according to one embodiment can match a first widget that is determined to be highly likely to be used with a selected first object. For example, a camper van object included in an image can be matched to a travel widget.

[0128] For example, the electronic device (101) can recommend a widget to the first object based on user information. For example, when a travel schedule included in the user's schedule approaches the current date, the electronic device (101) can recommend a schedule application widget to the first object and display information about the schedule through the widget.

[0129] For example, the electronic device (101) can recommend widgets based on personalized data. For example, the electronic device (1101) can match widgets suitable for a first object based on the user's location information or preferences (e.g., frequently used applications or recent search history). The electronic device (101) can recommend widgets to the first object based on user behavior prediction. The electronic device (101) can recommend widgets associated with the first object selected by the user, automatically place them on the home screen, or suggest grouping them with existing widgets. For example, if a camper van object is selected, it can recommend travel-related widgets and suggest grouping them with previously used travel-related applications.

[0130] In operation 950, an electronic device (101) according to one embodiment can generate a first background image by matching a first widget to a first object using a generative AI model. The electronic device (101) can generate a prompt requesting the generation of a background image that includes a first object to which a first widget is matched based on a first image, and input it into the generative AI model.

[0131] In operation 960, an electronic device (101) according to one embodiment may display a first wallpaper obtained through a generative AI model via a display. The electronic device (101) may display the first wallpaper in a preview form, may include a description (e.g., name) of a widget, and may provide an editing function for a component (e.g., widget or object) of the first wallpaper. The user may change the position, size, or shape of an object or recommended widget of the first wallpaper. The electronic device (101) may change the position of an app or widget, for example, by long-pressing. The electronic device (101) may move the widget to a desired location if drag-and-drop is input after a long-press input on the widget. The electronic device (101) may change the position so that it is naturally composited with the wallpaper after the move. The user may restore the previous edit through a specific action (e.g., cancel long press or reset settings). Upon restoration, the widget or icon may be harmoniously rearranged according to the style or layout of the wallpaper. The electronic device (101) can regenerate and display the first background image again by reflecting the changes when the position, size, or shape of an object or recommendation widget is changed by user input.

[0132] In operation 970, an electronic device (101) according to one embodiment may set the first background image as the background image of the electronic device (101) upon receiving user confirmation input for the first background image. If the user wishes to use the first background image that was automatically generated for the first image, the user may enter user confirmation, and if not, cancel or delete the first background image.

[0133] FIG. 10 is an example of an electronic device according to one embodiment of the present disclosure displaying a widget within a background screen.

[0134] An electronic device according to one embodiment (e.g., the electronic device (101) of FIG. 1) can display widgets within a background in various forms. In order to utilize objects within the background as widgets and icons, it may be necessary for the user to clearly recognize and distinguish them. The electronic device (101) can display the name of a widget or app by compositing it onto the object. The electronic device (101) can intuitively express the function of a widget by naturally compositing the widget name onto a specific object in the background as if it were written on the object itself. The electronic device (101) can apply a subtle shadow effect or an outline effect to the object. The electronic device (101) can express the object differently from other objects by adding a shadow effect behind the background object mapped to the widget or app, or by making it stand out from the background through the object's outline.

[0135] For example, in FIG. 10, the first background (1010) may include names for a calendar widget (1011), a clock app (1012), and a gallery app (1013). The second background (1020) may display a shadow effect on the borders of the calendar widget (1021), the clock app (1022), and the gallery app (1023). The third background (1030) may clearly display the outlines of the calendar widget (1031), the clock app (1032), and the gallery app (1033).

[0136] FIG. 11 is an example of changing the shape of a widget within a background screen using an electronic device according to one embodiment of the present disclosure.

[0137] An electronic device according to one embodiment (e.g., the electronic device (101) of FIG. 1) may display widget objects within a background differently. When an object included in an image set as a background is used as a widget or icon, the electronic device (101) may regenerate the background to move the related object to the position of the widget or icon that was previously included in the background, or to move it by the user's selection. The electronic device (101) may provide a function to dynamically customize the shape and basic form of a widget or app frame according to the style of a reference image selected by the user or the position of the background. For example, an existing rounded rectangular frame may be transformed into various shapes depending on the combination of the reference image or the background. The electronic device (101) may change the shape of the frame and widget object based on the reference image. The shape and frame of the widget and icon may be automatically transformed according to the style of the reference image selected by the user. For example, if an image with a soft drawing style is selected, the frame or shape may be transformed into a more rounded shape. Alternatively, if an image containing clouds is selected, the frame or shape may be transformed into a cloud shape. The electronic device (101) can change the frame and shape of an icon or widget based on the background. The electronic device (101) can automatically change the frame and shape by analyzing the location of the background where the app and widget are placed. For example, if the app and widget are placed in the sky area of ​​the background, the frame and shape may be transformed into a cloud shape. Or, if the app and widget are placed in the area with leaves on the background, a leaf-shaped shape or frame may be created. The electronic device (101) can create a widget object in a frameless form. If the user prefers frameless apps and widgets, the electronic device (101) can create only a form that reflects the style of a reference image. For example, the image object can be used as a widget as is.For example, cloud, star, or freeform widgets and app designs that simply match the style of the reference image can be applied without a frame.

[0138] The first screen (1110) may include a contact app (1113) in a table image (1111) that includes a flower vase (1112). The electronic device (101) may regenerate the shape of the app to match the background while changing the position of the app according to user input that changes the position of the app within the first screen (1110). For example, if the user moves the contact app to the flower bud of the flower vase (1112), the electronic device (101) may transform the shape of the contact app (1113) into a flower shape (1114). The electronic device (101) may change any one of the shape, color, pattern, or size of the app to reflect the background of the app's position.

[0139] The second screen (1120) may include a contact app (1123) in a table image (1121) that includes a flower vase (1122). When the user moves the contact app to the flower bud of the flower vase (1122), the electronic device (101) may match the contact app (1123) to the flower bud (1124) in a frameless form. At this time, to indicate that the flower bud (1124) is an app, the electronic device (101) may display an outer border line or a shadow effect (not shown) on the flower bud (1124) as in FIG. 11.

[0140] FIG. 12 is an example of an electronic device according to one embodiment of the present disclosure displaying a plurality of background images.

[0141] An electronic device according to one embodiment (e.g., the electronic device (101) of FIG. 1) can generate a plurality of wallpapers and display each wallpaper according to screen page transitions. The electronic device (101) can determine the configuration of the plurality of wallpapers (e.g., number of pages, type or number of components) based on user personal data (e.g., preferences). For example, if the electronic device (101) determines that a user's preference includes multiple widgets / icons on one page, it can place a relatively large number of widgets / icons on fewer pages. If the electronic device (101) determines that a user's preference includes only icons / widgets with high usage frequency on one page, it can distribute icons / widgets with a usage frequency above a threshold level across relatively many pages. The electronic device (101) can determine the page configuration differently by reflecting the user's situation. If the electronic device (101) determines that the user is busy with work, it can place icons / widgets with high usage frequency or high relevance to work on one page. When the electronic device (101) detects that the user is on vacation, it can place icons / widgets that are frequently used or highly relevant to vacation on multiple pages.

[0142] According to one embodiment, the electronic device (101) distinguishes and displays objects and widgets / icons within the background when moving between pages of the home screen, and can vary the state and display method of the app / widget objects for each page. The electronic device (101) can vary the display of objects for each page. For example, on the first page, the app / widget object may operate in an active state and be able to interact with the user. On the second page, the app / widget object from the first page may be displayed as a background in a deactivated state. Alternatively, the app / widget object from the first page may not be included in the background (is deleted), and another app / widget object from the second page may be displayed in that location.

[0143] An electronic device (101) according to one embodiment can switch the state of an app / widget between pages. When moving between pages, the electronic device (101) can disable the object on the first page and operate as a background, but maintain its display. The electronic device (101) can delete the object on the first page and activate and display another app / widget object set on the second page in that location.

[0144] An electronic device (101) according to one embodiment may maintain and display the objects on the first page as they are when there are no app / widget objects separately added by the user on the second page. When a new object is added by the user on the second page, the electronic device (101) may delete the objects on the first page and activate and display the objects set by the user on the second page.

[0145] According to one embodiment, when an electronic device (101) generates a wallpaper containing multiple pages as a single image, the configuration of each page may be different. For example, the objects included in each page may be configured differently, and the objects of the first page may be replaced with widgets or icons in the second page. Referring to FIG. 12, the first screen (1210) represents the first page.

[0146] The background of the first page may include a first object (1211), a second object (1212), a third widget (1213), or a fourth widget (1214). The first object (1211) and the second object (1212) are in the shape of a photo, and the third widget (1213) is matched with an SNS widget in the shape of a photo similar to the objects. The fourth widget (1214) is matched with a clock widget in the shape of a clock.

[0147] The second screen (1220) represents a second page. The second page includes a first object (1221) identical to the first object (1211) of the first page, a second object (1222) identical to the second object (1212), a third object (1223) which is a simple background with the same photo shape as the first widget (1213) of the first page but is not a widget, and may include a first widget (1224) which is a new widget. The fourth widget (1214) of the first page (e.g., a clock widget) may not be included in the second page.

[0148] The embodiments of this document and the terms used therein are not intended to limit the technical features described in this document to specific embodiments, and should be understood to include various modifications, equivalents, or substitutions of said embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of said items unless the relevant context clearly indicates otherwise. In this document, phrases such as "A or B," "at least one of A and B," "at least one of A or B," "A, B or C," "at least one of A, B and C," and "at least one of A, B, or C" each may include any one of the items listed together in the corresponding phrase, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used simply to distinguish said components from other said components and do not limit said components in any other aspect (e.g., importance or order). Where any (e.g., 1st) component is referred to as “coupled” or “connected” to another (e.g., 2nd) component, with or without the terms “functionally” or “communicationly,” it means that said any component may be connected to said other component directly (e.g., via a wire), wirelessly, or through a third component.

[0149] The term “module” as used in the embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit, for example. A module may be a component formed integrally, or a minimum unit of said component or a part thereof that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0150] One embodiment of the present document may be implemented as software (e.g., program (140)) comprising one or more instructions stored in a storage medium (e.g., internal memory (136) or external memory (138)) readable by a machine (e.g., electronic device (101)). For example, a processor (e.g., processor (120)) of the machine (e.g., electronic device (101)) may call at least one of the one or more instructions stored in the storage medium and execute it. This enables the machine to be operated to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code that can be executed by an interpreter. The storage medium readable by the machine may be provided in the form of a non-transitory storage medium. Here, 'non-temporary' simply means that the storage medium is a tangible device and does not contain a signal (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily.

[0151] According to one embodiment, the method according to the embodiments disclosed herein may be provided by being included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or distributed online (e.g., download or upload) through an application store (e.g., Play Store™) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created on a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0152] According to one embodiment, each component (e.g., module or program) of the components described above may include a singular or multiple entities, and some of the multiple entities may be separated and placed in other components. According to one embodiment, one or more of the components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Generally or additionally, multiple components (e.g., module or program) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each of the multiple components in the same or similar manner as those performed by the corresponding component among the multiple components prior to integration. According to one embodiment, operations performed by the module, program, or other components may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

Claims

1. In an electronic device, display; Memory for storing instructions; and It includes at least one processor including a processing circuit; When the above instructions are executed individually or collectively by the at least one processor, the electronic device: Receive input for a first image to be set as a wallpaper, and Recognizing at least one object and background included in the first image above, and Identifying the association between at least one widget or icon included in the previous background and the at least one object, and Based on the above association, the at least one widget or icon is matched to the at least one object, and A first background is generated using a generative AI model to include at least one matched widget or icon, and Displaying the above-mentioned first background image through the above-mentioned display, and An electronic device that causes the first wallpaper to be set as a wallpaper in response to receiving user confirmation for the first wallpaper.

2. In Paragraph 1, When the above instructions are executed individually or collectively by the at least one processor, the electronic device: In response to the determination that the first widget or icon among at least one widget or icon included in the above-mentioned previous background image and the first object among at least one object included in the above-mentioned first image have a high correlation above a threshold level, the first widget or icon is matched to the first object, and An electronic device that, in response to a determination that there is no object with a high correlation above a threshold level between a second widget or icon among at least one widget or icon included in the previous background image and at least one object included in the first image, generates a second object and causes the second widget or icon to be matched.

3. In Paragraph 1 or 2, When the above instructions are executed individually or collectively by the at least one processor, the electronic device: Request the above generative AI model to generate the above first background image based on personal data, and The above personal data comprises at least one of a user's profile, age, gender, language, social data, preferences, interests, event information, schedule information, location data, activity pattern information, activity data, usage history, search history, or gallery data, an electronic device.

4. In any one of paragraphs 1 through 3, When the above instructions are executed individually or collectively by the at least one processor, the electronic device: Identifying a first widget among a plurality of applications installed on the electronic device that is determined to have a higher probability of being used by a user than a threshold level based on personal data, and An electronic device that causes the first widget to be matched to a third object among the at least one object included in the first image that is determined to be associated with the first widget.

5. In any one of paragraphs 1 through 4, When the above instructions are executed individually or collectively by the at least one processor, the electronic device: Receiving user input for at least one widget or icon included in the first background image, and An electronic device that causes the position, size, shape, color, or name of at least one widget or icon to be changed according to the user input using the generative AI model.

6. In any one of paragraphs 1 through 5, When the above instructions are executed individually or collectively by the at least one processor, the electronic device: An electronic device that causes at least one widget or icon included in the first background screen to be placed on the first background screen by reflecting at least one of the position or size in the previous background screen.

7. In any one of paragraphs 1 through 6, When the above instructions are executed individually or collectively by the at least one processor, the electronic device: Acquire real-time data related to at least one widget included in the first background image, and Update the data of at least one widget according to the real-time data above, and Using the above generative AI model, regenerate a first background image that reflects at least one widget update data while maintaining visual consistency, and An electronic device that causes the first background image to be displayed.

8. In any one of paragraphs 1 through 7, When the above instructions are executed individually or collectively by the at least one processor, the electronic device: An electronic device that causes at least one object corresponding to at least one widget or icon included in the first background image to be displayed separately from the object included in the first background image.

9. In any one of paragraphs 1 through 8, When the above instructions are executed individually or collectively by the at least one processor, the electronic device: An electronic device that causes at least one of the name, shadow, or border of at least one of the above-mentioned widget or icon to be displayed.

10. In a method of an electronic device, The operation of receiving input for a first image to be set as a wallpaper; An operation to recognize at least one object and background included in the first image above; An action of confirming an association between at least one widget or icon included in the previous background and the at least one object; Based on the above association, an operation of matching the at least one widget or icon to the at least one object; The operation of generating a first wallpaper to include at least one matched widget or icon using a generative AI model; The operation of displaying the above-mentioned first background image through a display; and A method comprising the operation of setting the first wallpaper as a wallpaper in response to receiving user confirmation for the first wallpaper.

11. In Paragraph 10, Based on the above association, the operation of matching the at least one widget or icon to the at least one object is, An operation of matching the first widget or icon to the first object in response to a determination that the first widget or icon among at least one widget or icon included in the previous background image and the first object among at least one object included in the first image have a high correlation above a threshold level; and A method comprising the operation of creating a second object and matching the second widget or icon in response to the determination that there is no object with a high correlation level above a threshold level between the second widget or icon among at least one widget or icon included in the previous background image.

12. In Paragraph 10 or 11, The operation of generating a first wallpaper to include at least one matched widget or icon using the generative AI model above is, Request the above generative AI model to generate the above first background image based on personal data, and The above personal data comprises at least one of a user's profile, age, gender, language, social data, preferences, interests, event information, schedule information, location data, activity pattern information, activity data, usage history, search history, or gallery data.

13. In any one of paragraphs 10 through 12, An operation to identify a first widget among a plurality of applications installed on the electronic device, which is determined to have a higher probability of being used by a user than a threshold level based on personal data; and A method further comprising the operation of matching the first widget to a third object among the at least one object included in the first image that is determined to be associated with the first widget.

14. In any one of paragraphs 10 through 13, The operation of receiving user input for at least one widget or icon included in the first background image; and A method further comprising the action of changing the position, size, shape, color, or name of at least one widget or icon according to the user input using the generative AI model.

15. A non-transitory storage medium storing at least one instruction readable by a computer, When the above instructions are executed by at least one processor of the electronic device, the electronic device is caused to: The operation of receiving input for a first image to be set as a wallpaper; An operation to recognize at least one object and background included in the first image above; An action of confirming an association between at least one widget or icon included in the previous background and the at least one object; Based on the above association, an operation of matching the at least one widget or icon to the at least one object; The operation of generating a first wallpaper to include at least one matched widget or icon using a generative AI model; The operation of displaying the above-mentioned first background image through a display; and A storage medium that performs the operation of setting the first wallpaper as a wallpaper in response to receiving user confirmation for the first wallpaper.

Images