Electronic device and image generation method for electronic device

Abstract

This electronic device may comprise: a memory storing instructions and including one or more storage media; and at least one processor including processing circuitry. The instructions, when executed individually or collectively by the at least one processor, may control the electronic device to: convert a lock screen image set in the electronic device into a text form by using the processor or an artificial intelligence model; acquire information about a main object including any one of a person, a character, an article, and an animal and information about remaining objects; generate a prompt requesting generation of an image by using received notification content, the information about the main object, and the information about the remaining objects on the basis of receiving an application notification; generate a new image by providing the generated prompt as an input to the processor or the artificial intelligence model; and set the generated image as a lock screen image.

Description

Electronic devices and methods for generating images of electronic devices

[0001] This document relates to an electronic device and to a method for generating images (e.g., lock screen, wallpaper) using generative artificial intelligence.

[0002] Artificial Intelligence (AI) technology being applied to electronic devices has recently been developing rapidly. In particular, generative AI technology is being implemented in line with the evolution of AI capabilities. Generative AI refers to AI technology that utilizes machine learning and deep learning techniques to create new content similar to existing content, such as text, audio, and / or images. For example, generative AI can be applied to actions such as improving image quality, generating new images based on existing ones, or editing existing images.

[0003] Recently, with the rapid development of big data and deep-learning related technologies, artificial intelligence (AI) technology has been implemented in electronic devices and is also being applied to intelligent personal services that analyze specific data and provide and utilize integrated information from various fields specialized for users.

[0004] Recently, with the evolution of artificial intelligence technology, generative AI technology is being implemented. Generative AI can refer to AI technology that creates new, similar content using existing content such as text, audio, and / or images. For example, generative AI can refer to AI technology capable of generating content (e.g., text, audio, images, and / or videos) corresponding to a given input.

[0005] The above information may be provided as background art to aid in understanding the present disclosure. No claim or determination is made as to whether any of the above-described matters constitute prior art related to the present disclosure.

[0006] In one embodiment of the present disclosure, an electronic device that supports generative AI-based image editing (e.g., image extension, background generation, creation of objects within an image, reconstruction based on separation and / or addition), a method of operation, and a recording medium are provided.

[0007] In one embodiment of the present disclosure, an electronic device for editing an image and reconstructing (or regenerating) the image on a server or on a device based on the edited image, a method of operation thereof, and a recording medium are provided.

[0008] One embodiment proposes an electronic device, a method, and a recording medium capable of generating a prompt involved in image generation based on the characteristics of an object or background within an image, and regenerating an image based on the prompt.

[0009] An electronic device may include at least one processor storing instructions and including a memory and processing circuitry including one or more storage media. The instructions, when individually or collectively executed by the at least one processor, may cause the electronic device to convert a lock screen image set on the electronic device into text format using the processor or an artificial intelligence model, obtain information about a main object including any one of a person, a character object, and an animal and information about the remaining objects, generate a prompt requesting the generation of an image using the content of the received notification, the information about the main object, and the information about the remaining objects based on receiving an application notification, provide the generated prompt as an input to the processor or the artificial intelligence model to generate a new image, and control the generation of the generated image to be set as a lock screen image.

[0010] An operating method of an electronic device may include an operation of converting a lock screen image set on the electronic device into a text format using a processor or an artificial intelligence model, obtaining information on a main object including one of a person, a character object, and an animal and information on remaining objects, generating a prompt requesting generation of an image using the content of the received notification, the information on the main object, and the information on the remaining objects based on receiving an application notification, providing the generated prompt as an input to the processor or the artificial intelligence model to generate a new image, and setting the generated image as a lock screen image.

[0011] The electronic device according to this document can generate an image or video by reflecting information including application notifications or time in real time, and can enhance convenience so that the user can check the information at a glance.

[0012] The electronic device according to this document may determine the generation and placement of images differently based on the importance of the notification.

[0013] The electronic device according to this document can provide an image optimized for the user because it generates an object image similar to the features of an object based on an image containing an object (e.g., a person) through generative artificial intelligence.

[0014] FIG. 1 is a block diagram of an electronic device in a network environment according to various embodiments.

[0015] FIG. 2 is a block diagram showing an integrated intelligence system according to various embodiments.

[0016] FIG. 3 is a diagram showing the form in which relationship information between concepts and actions according to various embodiments is stored in a database.

[0017] FIG. 4 is a diagram schematically illustrating the configuration of an electronic device according to one embodiment of the present disclosure.

[0018] FIG. 5 illustrates a lock screen of an electronic device according to one embodiment.

[0019] FIG. 6 illustrates the process of generating a prompt and an image in response to a notification in an electronic device according to one embodiment.

[0020] FIG. 7 illustrates the process of generating a prompt and an image in response to a notification in an electronic device according to one embodiment.

[0021] FIG. 8 illustrates the process of generating a prompt and an image in response to a notification in an electronic device according to one embodiment.

[0022] FIG. 9 illustrates a user interface for controlling elements necessary for image generation in an electronic device according to one embodiment.

[0023] FIG. 10 is a flowchart illustrating a method for creating a lock screen of an electronic device according to one embodiment.

[0024] FIG. 11 is a flowchart illustrating a method for creating a lock screen of an electronic device according to one embodiment.

[0025] FIG. 1 is a block diagram of an electronic device (101) in a network environment (100) according to various embodiments. Referring to FIG. 1, in the network environment (100), the electronic device (101) may communicate with an electronic device (102) through a first network (198) (e.g., a short-range wireless communication network) or may communicate with at least one of an electronic device (104) or a server (108) through a second network (199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (101) may communicate with the electronic device (104) through a server (108). According to one embodiment, the electronic device (101) may include a processor (120), a memory (130), an input module (150), an audio output module (155), a display module (160), an audio module (170), a sensor module (176), an interface (177), a connection terminal (178), a haptic module (179), a camera module (180), a power management module (188), a battery (189), a communication module (190), a subscriber identification module (196), or an antenna module (197). In some embodiments, the electronic device (101) may omit at least one of these components (e.g., the connection terminal (178)), or may have one or more other components added. In some embodiments, some of these components (e.g., the sensor module (176), the camera module (180), or the antenna module (197)) may be integrated into one component (e.g., the display module (160)).

[0026] The processor (120) may, for example, execute software (e.g., a program (140)) to control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) and perform various data processing or operations. According to one embodiment, as at least a part of the data processing or operations, the processor (120) may store commands or data received from other components (e.g., a sensor module (176) or a communication module (190)) in a volatile memory (132), process the commands or data stored in the volatile memory (132), and store result data in a non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or an auxiliary processor (123) (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor) that can operate independently or together with the main processor (121). For example, when the electronic device (101) includes the main processor (121) and the auxiliary processor (123), the auxiliary processor (123) may be configured to use less power than the main processor (121) or to be specialized for a given function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as a part thereof.

[0027] The auxiliary processor (123) may control at least a portion of functions or states associated with at least one component (e.g., a display module (160), a sensor module (176), or a communication module (190)) of the electronic device (101), for example, on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (123) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (180) or a communication module (190)). In one embodiment, the auxiliary processor (123) (e.g., a neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. This learning can be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or can be performed through a separate server (e.g., server (108)). The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include multiple artificial neural network layers.The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to, or alternatively to, a hardware structure, an artificial intelligence model may include a software structure.

[0028] The memory (130) can store various data used by at least one component (e.g., processor (120) or sensor module (176)) of the electronic device (101). The data can include, for example, software (e.g., program (140)) and input data or output data for commands related thereto. The memory (130) can include volatile memory (132) or non-volatile memory (134).

[0029] The program (140) may be stored as software in the memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0030] The input module (150) can receive commands or data to be used in a component of the electronic device (101) (e.g., a processor (120)) from an external source (e.g., a user) of the electronic device (101). The input module (150) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0031] The audio output module (155) can output audio signals to the outside of the electronic device (101). The audio output module (155) can include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. In one embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

[0032] The display module (160) can visually provide information to an external (e.g., user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling said device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of the force generated by said touch.

[0033] The audio module (170) can convert sound into an electrical signal, or vice versa, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150), output sound through the sound output module (155), or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphone) directly or wirelessly connected to the electronic device (101).

[0034] The sensor module (176) can detect the operating status (e.g., power or temperature) of the electronic device (101) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. According to one embodiment, the sensor module (176) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0035] The interface (177) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (101) with an external electronic device (e.g., the electronic device (102)). In one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0036] The connection terminal (178) may include a connector through which the electronic device (101) may be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0037] The haptic module (179) can convert electrical signals into mechanical stimuli (e.g., vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic sensations. According to one embodiment, the haptic module (179) can include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

[0038] The camera module (180) can capture still images and videos. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0039] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

[0040] A battery (189) may power at least one component of the electronic device (101). In one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0041] The communication module (190) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may operate independently from the processor (120) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (194) (e.g., a local area network (LAN) communication module, or a power line communication module). Among these communication modules, the corresponding communication module can communicate with an external electronic device (104) via a first network (198) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (199) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules can be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can verify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) by using subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (196).

[0042] The wireless communication module (192) can support 5G networks and next-generation communication technologies following the 4G network, such as NR access technology (new radio access technology). The NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and connection of multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency communications)). The wireless communication module (192) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The wireless communication module (192) can support various technologies for securing performance in the high-frequency band, such as beamforming, massive MIMO (multiple-input and multiple-output), full-dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large-scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), external electronic device (e.g., electronic device (104)), or network system (e.g., second network (199)). According to one embodiment, the wireless communication module (192) can support a peak data rate (e.g., 20 Gbps or more) for eMBB realization, a loss coverage (e.g., 164 dB or less) for mMTC realization, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL), or 1 ms or less for round trip) for URLLC realization.

[0043] The antenna module (197) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). In one embodiment, the antenna module (197) may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). In one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (198) or the second network (199), may be selected from the plurality of antennas, for example, by the communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device via the at least one selected antenna. In some embodiments, in addition to the radiator, another component (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module (197).

[0044] According to various embodiments, the antenna module (197) may form a mmWave antenna module. In one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent a first side (e.g., a bottom side) of the printed circuit board and capable of supporting a designated high-frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) disposed on or adjacent a second side (e.g., a top side or a side side) of the printed circuit board and capable of transmitting or receiving signals in the designated high-frequency band.

[0045] At least some of the above components can be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)).

[0046] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) via a server (108) connected to a second network (199). Each of the external electronic devices (102 or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations executed in the electronic device (101) may be executed in one or more of the external electronic devices (102, 104, or 108). For example, when the electronic device (101) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (101) may, instead of or in addition to executing the function or service itself, request one or more external electronic devices to perform the function or at least a part of the service. One or more external electronic devices that receive the request may execute at least a portion of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may process the result as is or additionally and provide it as at least a portion of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used, for example. The electronic device (101) may provide an ultra-low latency service by using distributed computing or mobile edge computing, for example. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server utilizing machine learning and / or a neural network. According to one embodiment, the external electronic device (104) or the server (108) may be included in the second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0047] The electronic device according to the various embodiments disclosed in this document may be of various forms. The electronic device may include, for example, a portable communication device (e.g., a smartphone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a consumer electronics device. The electronic device according to the embodiments of this document is not limited to the devices described above.

[0048] The various embodiments of this document and the terminology used therein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly indicates otherwise. In this document, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" can include any one of the items listed together in the corresponding phrase among those phrases, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used merely to distinguish one component from another, and do not limit the components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as "coupled" or "connected" to another component (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0049] The term "module" used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or part of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0050] Various embodiments of the present document may be implemented as software (e.g., a program (140)) including one or more instructions stored in a storage medium (e.g., an internal memory (136) or an external memory (138)) readable by a machine (e.g., an electronic device (101)). For example, a processor (e.g., a processor (120)) of the machine (e.g., an electronic device (101)) may call at least one instruction among the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' simply means that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently or temporarily on the storage medium.

[0051] According to one embodiment, the method according to various embodiments disclosed in this document may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) through an application store (e.g., Play Store™) or directly between two user devices (e.g., smart phones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0052] According to various embodiments, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separated and placed in other components. According to various embodiments, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to various embodiments, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

[0053] FIG. 2 is a block diagram showing an integrated intelligence system according to one embodiment.

[0054] Referring to FIG. 2, an integrated intelligent system according to one embodiment may include an electronic device (210) (e.g., the electronic device (101) of FIG. 1), an intelligent server (230) (e.g., the server (108) of FIG. 1), and a service server (250) (e.g., the server (108) of FIG. 1).

[0055] According to one embodiment, the electronic device (210) may be a terminal device (or electronic device) capable of connecting to the Internet, and may be, for example, a mobile phone, a smartphone, a PDA (personal digital assistant), a laptop computer, a TV, a white goods appliance, a wearable device, a HMD (head mounted display), or a smart speaker.

[0056] According to the illustrated embodiment, the electronic device (210) may include a communication interface (213) (e.g., interface (177) of FIG. 1), a microphone (212) (e.g., input module (150) of FIG. 1), a speaker (216) (e.g., sound output module (155) of FIG. 1), a display module (211) (e.g., display module (160) of FIG. 1), a memory (215) (e.g., memory (130) of FIG. 1), or a processor (214) (e.g., processor (120) of FIG. 1). The listed components may be operatively or electrically connected to each other. The electronic device (210) may include at least some of the configurations and / or functions of the electronic device (101) of FIG. 1.

[0057] According to one embodiment, the communication interface (213) may be configured to be connected to an external device to transmit and receive data.

[0058] According to one embodiment, the display module (211) may be configured to display an image or video. According to one embodiment, the display module (211) may also display a graphic user interface (GUI) of an app (or application program) being executed. The display module (211) of one embodiment may receive touch input through a touch sensor. For example, the display module (211) may receive text input through a touch sensor in an image keyboard area displayed within the display module (211).

[0059] According to one embodiment, memory (215) may store a client module (218), an SDK (software development kit) (217), and a plurality of apps (e.g., a first app (219a), a second app (219b)). The client module (218) and the SDK (217) may form a framework (or solution program) for performing general-purpose functions. Additionally, the client module (218) or the SDK (217) may form a framework for processing user input (e.g., voice input, text input, touch input).

[0060] According to one embodiment, the plurality of apps (219a, 219b) stored in memory (215) may be programs for performing a designated function. According to one embodiment, the plurality of apps may include a first app (219a) and a second app (219b). According to one embodiment, each of the plurality of apps (219a, 219b) may include a plurality of operations for performing a designated function. For example, the apps (219a, 219b) may include an alarm app, a message app, and / or a schedule app. According to one embodiment, the plurality of apps (219a, 219b) may be executed by a processor (214) to sequentially execute at least some of the plurality of operations.

[0061] According to one embodiment, the processor (214) can control the overall operation of the electronic device (210). For example, the processor (214) can be electrically connected to a communication interface (213), a microphone (212), a speaker (216), and a display module (211) to perform a specified operation.

[0062] According to one embodiment, the processor (214) may also execute a program stored in the memory (215) to perform a designated function. For example, the processor (214) may execute at least one of the client module (218) or the SDK (217) to perform the following operations for processing user input. The processor (214) may control the operations of a plurality of apps (219a, 219b), for example, through the SDK (217). The following operations described as operations of the client module (218) or the SDK (217) may be operations executed by the processor (214).

[0063] According to one embodiment, the client module (218) can receive user input. The client module (218) can receive touch input detected through the display module (211). Alternatively, the client module (218) can receive text input detected through a keyboard or a screen keyboard. In addition, the client module (218) can receive various types of user input detected through an input module included in the electronic device (210) or an input module connected to the electronic device (210). The client module (218) can transmit the received user input to the intelligent server (230). The client module (218) can transmit status information of the electronic device (210) together with the received user input to the intelligent server (230). The status information can be, for example, execution status information of an app.

[0064] According to one embodiment, the client module (218) can receive a result corresponding to the received user input. For example, the client module (218) can receive a result corresponding to the received user input if the intelligent server (230) can produce a result corresponding to the received user input. The client module (218) can display the received result on the display module (211). Additionally, the client module (218) can output the received result as audio through the speaker (216).

[0065] According to one embodiment, the client module (218) may receive a plan corresponding to the received user input. The client module (218) may display the results of executing a plurality of actions of the app according to the plan on the display module (211). The client module (218) may, for example, sequentially display the results of executing a plurality of actions on the display module (211) and output audio through the speaker (216). The electronic device (210) may, for another example, display only some of the results of executing a plurality of actions (e.g., the result of the last action) on the display module (211) and output audio through the speaker (216).

[0066] In one embodiment, the client module (218) may receive a request from the intelligent server (230) to obtain information necessary to produce a result corresponding to the prompt. In one embodiment, the client module (218) may transmit the necessary information to the intelligent server (230) in response to the request.

[0067] According to one embodiment, the client module (218) may transmit result information of executing multiple operations according to a plan to the intelligent server (230). The intelligent server (230) may use the result information to confirm that the received user input has been processed correctly.

[0068] In one embodiment, the intelligent server (230) may receive information related to a prompt from the electronic device (210) via a communication network. In one embodiment, the intelligent server (230) may generate a plan for performing a corresponding task based on the data related to the received prompt.

[0069] According to one embodiment, a plan may be generated by an artificial intelligence (AI) system. The AI ​​system may be a rule-based system or a neural network-based system (NNS) (e.g., a feedforward neural network (FNN), a recurrent neural network (RNN)). Alternatively, it may be a combination of the foregoing or an AI system other than the above. According to one embodiment, the plan may be selected from a set of predefined plans or may be generated in real time in response to a user request. For example, the AI ​​system may select at least one plan from a plurality of predefined plans.

[0070] According to one embodiment, the intelligent server (230) may transmit the result according to the generated plan to the electronic device (210) or transmit the generated plan to the electronic device (210). According to one embodiment, the electronic device (210) may display the result according to the plan on the display module (211). According to one embodiment, the electronic device (210) may display the result of executing the operation according to the plan on the display module (211).

[0071] According to one embodiment, the intelligent server (230) may include a front end (231), a natural language platform (232), a capsule database (238), an execution engine (233), an end user interface (234), a management platform (235), a big data platform (236), and / or an analytic platform (237).

[0072] According to one embodiment, the front end (231) can receive user input received from the electronic device (210). The front end (231) can transmit a response corresponding to the user input.

[0073] According to one embodiment, the natural language platform (232) may include an automatic speech recognition module (ASR module) (232a), a natural language understanding module (NLU module) (232b), a planner module (232c), a natural language generator module (NLG module) (232d), and / or a text to speech module (TTS module) (232e).

[0074] According to one embodiment, the natural language understanding module (232b) can determine the user's intent by performing syntactic analysis and / or semantic analysis on user input in the form of text data. According to one embodiment, the natural language understanding module (232b) can determine the meaning of a word within a prompt by using linguistic features (e.g., grammatical elements) of a morpheme or phrase, and determine the user's intent by matching the determined meaning of the word to the intent. The natural language understanding module (223b) can obtain intent information corresponding to the user's utterance. The intent information may be information indicating the user's intent determined by interpreting the text data. The intent information may include information indicating an action or function that the user intends to execute using the device.

[0075] According to one embodiment, the planner module (232c) can generate a plan using the intent and parameters determined by the natural language understanding module (232b). According to one embodiment, the planner module (232c) can determine a plurality of domains necessary to perform a task based on the determined intent. The planner module (232c) can determine a plurality of actions included in each of the plurality of domains determined based on the intent. According to one embodiment, the planner module (232c) can determine parameters necessary to execute the determined plurality of actions or result values ​​output by the execution of the plurality of actions. The parameters and the result values ​​may be defined as concepts of a specified format (or class). Accordingly, the plan may include a plurality of actions and a plurality of concepts determined by the user's intent.

[0076] According to one embodiment, the planner module (232c) can determine the relationship between the plurality of operations and the plurality of concepts in a stepwise (or hierarchical) manner. For example, the planner module (232c) can determine the execution order of the plurality of operations determined based on the user's intention based on the plurality of concepts. In other words, the planner module (232c) can determine the execution order of the plurality of operations based on parameters required for the execution of the plurality of operations and results output by the execution of the plurality of operations. Accordingly, the planner module (232c) can generate a plan that includes association information (e.g., ontology) between the plurality of operations and the plurality of concepts. The planner module (232c) can generate the plan using information stored in a capsule database (238) in which a set of relationships between concepts and operations is stored.

[0077] According to one embodiment, the natural language generation module (232d) can change specified information into a text form. The information changed into a text form may be in the form of a natural language utterance. According to one embodiment, the text-to-speech conversion module (232e) can change information in a text form into information in a speech form.

[0078] According to one embodiment, some or all of the functions of the natural language platform (232) may also be implemented in the electronic device (210).

[0079] The capsule database (238) can store information regarding the relationships between multiple concepts and actions corresponding to multiple domains. A capsule according to one embodiment may include multiple action objects (or action information) and concept objects (or concept information) included in a plan. According to one embodiment, the capsule database (238) may store multiple capsules in the form of a CAN (concept action network). According to one embodiment, multiple capsules may be stored in a function registry included in the capsule database (238).

[0080] According to one embodiment, the execution engine (233) can produce a result using the generated plan. The end user interface (234) can transmit the produced result to the electronic device (210). Accordingly, the electronic device (210) can receive the result and provide the received result to the user. According to one embodiment, the management platform (235) can manage information used in the intelligent server (230). According to one embodiment, the big data platform (236) can collect user data. According to one embodiment, the analysis platform (237) can manage the quality of service (QoS) of the intelligent server (230). For example, the analysis platform (237) can manage the components and processing speed (or efficiency) of the intelligent server (230).

[0081] According to one embodiment, the service server (250) may be a server operated by a third party. According to one embodiment, the service server (250) may provide information to the intelligent server (230) for generating a plan corresponding to a received prompt. The provided information may be stored in a capsule database (238). Additionally, the service server (250) may provide result information according to the plan to the intelligent server (230). The service server (250) may include a plurality of service providers (e.g., CP service A (251), CP service B (252), CP service C (253)), and each service provider (251, 252, 253) may provide functions for a domain associated with each capsule stored in the capsule database (238) of the intelligent server (230).

[0082] In the integrated intelligent system described above, the electronic device (210) can provide various intelligent services to the user in response to user input. The user input may include, for example, input via a physical button, touch input, or input via a prompt.

[0083] According to one embodiment, when an electronic device (210) provides services together with an intelligent server (230) and / or a service server (250), the electronic device (210) may generate an image (or video) corresponding to a prompt. The electronic device (210) may transmit the generated image (or video) to the intelligent server (230) via a network (240) using a communication interface (213).

[0084] According to one embodiment, the electronic device (210) can receive the response using a communication interface (213). The electronic device (210) can output a voice signal generated inside the electronic device (210) to the outside using the speaker (216), or output an image generated inside the electronic device (210) to the outside using a display module (211).

[0085] In FIG. 2, an example is described in which natural language understanding and generation, and the operation of producing results using a plan, are performed on an intelligent server (230) in an electronic device (210), but various embodiments of this document are not limited thereto. For example, at least some components of the intelligent server (230) (e.g., natural language platform (232), execution engine (233), capsule database (238)) may be embedded in the electronic device (210) (or the electronic device (101) of FIG. 1), and the operation may be performed by the electronic device (210).

[0086] FIG. 3 is a diagram showing the form in which relationship information between concepts and operations is stored in a database according to various embodiments.

[0087] According to one embodiment, a capsule database (e.g., capsule database (238) of FIG. 2) of an intelligent server (e.g., intelligent server (230) of FIG. 2) may store capsules in the form of a CAN (concept action network) (300). The capsule database may store an action for processing a task corresponding to an input prompt, and parameters required for said action, in the form of a CAN (concept action network).

[0088] According to one embodiment, the capsule database may store a plurality of capsules (capsule (A) (310), capsule (B) (320)) corresponding to each of a plurality of domains (e.g., applications). According to one embodiment, one capsule (e.g., capsule (A) (310)) may correspond to one domain (e.g., location (geo), application). Additionally, one capsule may correspond to at least one service provider (e.g., CP 1 (331) or CP 2 (332)) for performing functions for the domain associated with the capsule. According to one embodiment, one capsule may include at least one operation (350) and at least one concept (360) for performing a designated function.

[0089] According to one embodiment, a natural language platform (e.g., the natural language platform (232) of FIG. 2) can generate a plan for performing a task corresponding to a received prompt using capsules stored in a capsule database. For example, a planner module of the natural language platform (e.g., the planner module (232c) of FIG. 2) can generate a plan using capsules stored in a capsule database. For example, a plan can be generated using the actions (311, 313) and concepts (312, 314) of capsule A (310) and the actions (321) and concepts (322) of capsule B (320).

[0090] FIG. 4 is a diagram schematically illustrating the configuration of an electronic device according to one embodiment of the present disclosure.

[0091] Referring to FIG. 4, an electronic device (101) according to one embodiment of the present disclosure may include a display (490) (e.g., a display module (160, 211) of FIG. 1 or 2), a memory (130) (e.g., a memory (130) of FIG. 1), a communication circuit (495) (e.g., a communication module (190) or a communication interface (213) of FIG. 1) and / or a processor (120) (e.g., a processor (120, 214) of FIG. 1 or 2). According to one embodiment, the electronic device (101) may include all or at least some of the components of the electronic device (101, 210) as described in the description portion with reference to FIG. 1 or 2. For example, in various embodiments of this document, some of the illustrated components may be omitted or substituted. The electronic device (101) may include at least some of the configurations and / or functions of the electronic device (101) of FIG. 1 and / or the electronic device (210) of FIG. 2. At least some of the configurations of each of the electronic device (101) shown (or not shown) may be operatively, functionally, and / or electrically connected.

[0092] According to one embodiment, the display (490) may include a configuration identical or similar to the display module (160, 211) of FIG. 1 or FIG. 2. According to one embodiment, the display (490) may display various images provided by the processor (120). According to one embodiment, the display (490) may visually provide various screens related to an application being executed (e.g., the application (146) of FIG. 1) and its use (e.g., contents screen, application execution screen, menu screen, and / or function execution screen) under the control of the processor (120).

[0093] According to one embodiment, the display (490) may be combined with a touch sensor, a pressure sensor capable of measuring the intensity of the touch, and / or a touch panel (e.g., a digitizer) that detects a magnetic field-based stylus pen. According to one embodiment, the display (490) may detect touch input, air gesture input, and / or hovering input (or proximity input) by measuring a change in a signal (e.g., voltage, light intensity, resistance, electromagnetic signal, and / or charge quantity) at a specific location on the display (490) based on the touch sensor, pressure sensor, and / or touch panel. For example, the display (490) may include a touchscreen that detects touch and / or proximity touch (or hovering) input using a part of the user's body (e.g., a finger) or an input device (e.g., a stylus pen). The display (490) may include at least some of the configurations and / or functions of the display module (160) of FIG. 1 and / or the display module (211) of FIG. 2.

[0094] According to one embodiment, the display (490) may include, but is not limited to, a liquid crystal display (LCD), a light-emitting diode (LED), an organic light-emitting diode (OLED) display, and / or an active matrix OLED (AMOLED) display, a micro electro mechanical systems (MEMS) display, or an electronic paper display. According to one embodiment, the display (490) may include a flexible display.

[0095] According to one embodiment, the memory (130) includes at least a portion of the configuration and / or function of the memory (130) of FIG. 1 and / or the memory (215) of FIG. 2, and may store software (e.g., the program (140) of FIG. 1). The memory (101) may store various applications (e.g., the application (146) of FIG. 1 or the plurality of apps (219a, 219b) of FIG. 2), and program modules supporting intelligent services (e.g., the client module (218) of FIG. 2).

[0096] According to one embodiment, the memory (130) may store various data used by at least one component (e.g., processor (120)) of the electronic device (101). In one embodiment, the data may include, for example, software (e.g., program (140) of FIG. 1), and input data or output data for commands related to the software.

[0097] According to one embodiment, the memory (130) may include volatile memory (e.g., volatile memory (132) of FIG. 1) or non-volatile memory (134) (e.g., non-volatile memory (134) of FIG. 1). According to one embodiment, the memory (130) may store instructions or data received from the processor (120) in the volatile memory (1320), and may store result data of instructions or data stored in the volatile memory (132) being processed by the processor (120) in the non-volatile memory (134).

[0098] In one embodiment, the data may include various data (e.g., training data, prompt data, context awareness data, and / or urgency (or priority) determination data) to support the electronic device (101) in controlling the operation of providing artificial intelligence-based notification information. In one embodiment, the data may include information regarding various settings to support the electronic device (101) in controlling the operation of providing artificial intelligence-based notification information.

[0099] In one embodiment, the data may include various learning data and / or parameters obtained based on the user's learning through interaction with the user. In one embodiment, the data may include various schemas (or algorithms, models, networks, or functions) to support an operation of providing AI-based notification information.

[0100] For example, a schema for supporting the operation of providing artificial intelligence-based notification information in an electronic device (101) may include a neural network. In one embodiment, the neural network may include a neural network model based on at least one of an artificial neural network (ANN), a convolutional neural network (CNN), a region with convolutional neural network (R-CNN), a region proposal network (RPN), a recurrent neural network (RNN), a stacking-based deep neural network (S-DNN), a state-space dynamic neural network (S-SDNN), a Deconvolution Network, a deep belief network (DBN), a restricted Boltzmann machine (RBM), a long short-term memory (LSTM) network, a classification network, a plain residual network, a dense network, a hierarchical pyramid network, and / or a fully convolutional network. According to one embodiment, the type of neural network model is not limited to the examples described above.

[0101] According to one embodiment, the memory (130) may store instructions that cause the processor (120) to operate when executed. For example, the instructions may be stored in the memory (130) as software (e.g., program (140) of FIG. 1) and may be executed by the processor (120). For example, the instructions may include control commands such as arithmetic and logical operations, data movement, and / or input / output that can be recognized by the processor (120). According to one embodiment, the software may include various applications (e.g., application (146) of FIG. 1) that can provide various functions (or services) (e.g., call function, message function, messenger function, email function, social networking service (SNS) function, search function, media (e.g., video and / or music) playback function, game function, and / or wireless communication function) in the electronic device (101).

[0102] According to one embodiment, the communication circuit (495) may support the establishment of a designated wireless communication channel (e.g., short-range communication such as Bluetooth communication and / or BLE communication) and the performance of communication through the established wireless communication channel. For example, the communication circuit (495) may perform designated communication (e.g., Bluetooth communication and / or BLE communication) with an external device. According to one embodiment, the communication circuit (495) may support wireless communication with an external device using cellular wireless communication (e.g., 4G LTE, 5G NR) and / or short-range wireless communication (e.g., Wi-Fi). For example, the electronic device (101) may use the communication circuit (495) to communicate with an external server (e.g., a generative AI server) that provides AI-based functions (or services) over a network. According to one embodiment, the communication circuit (495) may transmit data (e.g., images and / or prompts) generated from the electronic device (101) to an external server and receive data (e.g., images) transmitted from the external server. According to one embodiment, the communication circuit (495) may include at least some of the configuration and / or functionality of the communication module (190) of FIG. 1 and / or the communication interface (210) of FIG. 2.

[0103] According to one embodiment, the processor (120) can perform application layer processing functions required by a user of the electronic device (101). According to one embodiment, the processor (120) can provide control and commands for functions for various blocks of the electronic device (101). According to one embodiment, the processor (120) can perform operations or data processing regarding the control and / or communication of each component of the electronic device (101).

[0104] According to one embodiment, the processor (120) may include one or more processors comprising processing circuitry and / or executable program elements. According to one embodiment, the processor (120) may control (or process) overall operations related to generating and providing context-appropriate notification information based on artificial intelligence (AI), based on processing circuitry and / or executable program elements.

[0105] According to one embodiment, the processor (120) can perform an operation of performing a task related to an application. According to one embodiment, the processor (120) can perform an operation of detecting a notification while performing the task. According to one embodiment, the processor (120) can perform an operation of performing context awareness related to the task based on detecting a notification. According to one embodiment, the processor (120) can perform an operation of extracting data for a notification interface based on the situation resulting from the context awareness. According to one embodiment, the processor (120) can perform an operation of creating a notification interface corresponding to the situation based on the extracted data. According to one embodiment, the processor (120) can perform an operation of providing a notification interface associated with the task.

[0106] According to one embodiment, the processor (120) may be an application processor (AP). According to one embodiment, the processor (120) may be a system semiconductor responsible for computation and multimedia driving functions of an electronic device (101). According to one embodiment, the processor (120) may be configured in the form of a system-on-chip (SoC) and may include a technology-intensive semiconductor chip that integrates various semiconductor technologies into one and implements system blocks into a single chip.

[0107] According to one embodiment, the system blocks of the processor (120) may include a graphics processing unit (GPU) (410), an image signal processor (ISP) (420), a central processing unit (CPU) (430), a neural processing unit (NPU) (440), a digital signal processor (DSP) (450), a modem (460), a connectivity (470), and / or a security (480) block, as illustrated in FIG. 4.

[0108] According to one embodiment, the GPU (410) may be responsible for graphics processing. According to one embodiment, the GPU (410) may receive instructions from the CPU (430) and perform graphics processing to represent the shape, position, color, shading, movement, and / or texture of objects (or things) on a display.

[0109] In one embodiment, the ISP (420) may be responsible for image processing and correction of images and videos. In one embodiment, the ISP (420) may correct raw data (e.g., raw data) transmitted from an image sensor of a camera (e.g., the camera module (180) of FIG. 1) to generate an image in a form more preferred by the user. In one embodiment, the ISP (420) may perform post-processing, such as adjusting partial brightness of an image and emphasizing detailed parts. For example, the ISP (420) may independently perform a process of tuning and correcting the image quality of an image acquired through a camera to generate a result preferred by the user.

[0110] According to one embodiment, the ISP (420) may support artificial intelligence (AI)-based image processing technology. According to one embodiment, the ISP (420) may support scene segmentation (e.g., image segmentation) technology that recognizes and / or classifies parts of a scene being captured in conjunction with the NPU (440). For example, the ISP (420) may include a function that applies different parameters to objects such as the sky, bushes, and / or skin and processes them. According to one embodiment, the ISP (420) may detect and display a human face during image capture using the AI ​​function, or adjust the brightness, focus, and / or color of the image using the coordinates and information of the face.

[0111] According to one embodiment, the CPU (430) may perform a role corresponding to the processor (120). According to one embodiment, the CPU (430) may decode user commands and perform arithmetic and logical operations, and / or data processing. For example, the CPU (430) may perform functions such as memory, interpretation, computation, and control. According to one embodiment, the CPU (430) may control the overall functions of the electronic device (101). For example, the CPU (430) may execute all software of the electronic device (101) (e.g., the application (146) of FIG. 1 and / or a plurality of apps (219a, 219b) of FIG. 2) on an operating system (OS) and control hardware devices. According to one embodiment, the CPU (430) may execute the application and control the overall operation of the processor (120) to perform neural network-based tasks required according to the execution of the application.

[0112] According to one embodiment, the CPU (430) may store instructions or data in the volatile memory of the memory (130) (e.g., the volatile memory (132) of FIG. 1) as at least part of data processing or computation, process the instructions or data stored in the volatile memory, and store the result data in the non-volatile memory of the memory (130) (e.g., the non-volatile memory (134) of FIG. 1).

[0113] According to one embodiment, the CPU (430) may include a single processor core or a plurality of processor cores. According to one embodiment, the CPU (430) may be a programmable processor capable of executing instructions (e.g., instructions capable of performing operations of the CPU (430)).

[0114] According to one embodiment, the CPU (430) may operate in a multi-domain environment. According to one embodiment, the CPU (430) may operate in a domain of a normal world (e.g., a non-secure world, a framework, or a non-secure environment) and a multi-domain environment of a secure world (e.g., a security framework or a security environment). In one embodiment, the domain of the secure world may include one or more domains (e.g., a trusted OS, a Trustzone, and / or a virtualization framework).

[0115] According to one embodiment, the NPU (440) may be responsible for processing optimized for deep-learning algorithms of artificial intelligence. According to one embodiment, the NPU (440) is a processor optimized for deep-learning algorithm computations (e.g., artificial intelligence computations) and can process big data quickly and efficiently like a human neural network. For example, the NPU (440) may be primarily used for artificial intelligence computations. According to one embodiment, the NPU (440) may be responsible for processing that automatically adjusts the focus by recognizing objects, environments, and / or people in the background when shooting video through a camera, automatically switches the shooting mode of the camera module (180) to food mode when taking food photos, and / or removes only unnecessary subjects from the captured results. According to one embodiment, the NPU (440) may be responsible for processing that analyzes various situations based on a running application based on situation awareness and provides notifications in accordance with the situation corresponding to the analysis results.

[0116] According to one embodiment, the electronic device (101) can support integrated machine learning processing by interacting with all processors such as a GPU (410), an ISP (420), a CPU (430), and an NPU (440).

[0117] In one embodiment, the DSP (450) may represent an integrated circuit that facilitates rapid processing of digital signals. In one embodiment, the DSP (450) may perform the function of converting analog signals into digital signals and performing high-speed processing.

[0118] According to one embodiment, the modem (460) may perform the role of enabling various communication functions in the electronic device (101). For example, the modem (460) may support communication such as telephone and data transmission and reception while exchanging signals with a base station. According to one embodiment, the modem (460) may include an integrated modem (e.g., cellular modem, LTE modem, 5G modem, 5G-Advanced modem, and 6G modem) that supports communication technologies such as LTE (long term evolution) and 2G to 5G. According to one embodiment, the modem (460) may include an artificial intelligence modem that applies an artificial intelligence algorithm.

[0119] According to one embodiment, the connectivity (470) may support wireless data transmission based on IEEE 802.11. According to one embodiment, the connectivity (470) may support communication services based on IEEE 802.11 (e.g., Wi-Fi) and / or 802.15 (e.g., Bluetooth, ZigBee, UWB). For example, the connectivity (470) may support communication services for an unspecified number of people in a localized area, such as indoors, using an unlicensed band.

[0120] FIG. 5 illustrates a lock screen of an electronic device according to one embodiment.

[0121] In Figure 510, an electronic device (e.g., electronic device (101) of Figure 1) can analyze a set lock screen using a processor (e.g., processor (120) of Figure 1) or generative artificial intelligence.

[0122] According to one embodiment, the generative AI is a type of AI model and may be placed inside an electronic device (101) or on an external server. According to one embodiment, a processor (120) may provide the generative AI to execute an image generation process based on generated instructions. According to one embodiment, the generated instructions may be provided to the generative AI on the device and / or the generative AI on the server.

[0123] According to one embodiment, an electronic device (101) can analyze a lock screen set by a user and convert it into text. The electronic device (101) can provide an image of the set lock screen as input to a generative artificial intelligence and also input a prompt requesting that a description of the image be expressed in text. The electronic device (101) can convert the set lock screen into text using the generative artificial intelligence. The electronic device (101) can analyze the lock screen set by the user using a processor (120) or a generative artificial intelligence. The electronic device (101) can generate a prompt requesting that the lock screen set by the user be analyzed and converted into text. The electronic device (101) can analyze the set lock screen by providing the generated prompt as input to the generative artificial intelligence. Based on the analysis results, the electronic device (101) can separate and recognize the main objects and remaining objects of the set lock screen and distinguish the type of the lock screen. The type of the lock screen can be divided into whether it is an image containing a person, an image of a landscape, or an image containing an object. The main object may include, for example, at least one of a person, animal, character, or object. The remaining object may refer to objects other than people, animals, characters, and objects (e.g., landscape, background). The type of lock screen is merely an example and is not limited thereto. The type of object is also merely an example and is not limited thereto. The electronic device (101) may set the main object as the 'subject' and the remaining object as the 'background' to generate a prompt requesting the creation of an image.

[0124] According to one embodiment, the electronic device (101) may determine that the set lock screen image is an image of a background based on the result of analyzing the set lock screen image and finding that a main object is not displayed in the display area. The main object mentioned below may include, for example, any one of a person, a character, an object, and an animal. The types of main objects are merely examples and are not limited thereto.

[0125] Alternatively, the electronic device (101) may determine that the set lock screen image is an image for the background based on the size or ratio of the area occupied by the main object being below a specified level.

[0126] The electronic device (101) may identify a main object in a display area and determine that a set lock screen image is an image of the object based on the size of the area occupied by the identified main object being greater than or equal to a specified level. Alternatively, the electronic device (101) may identify a main object and determine that a set lock screen image is an image of the object based on the ratio of the area occupied by the identified main object being greater than or equal to a specified level.

[0127] According to one embodiment, an electronic device (101) can analyze a lock screen image set by a user to determine whether a major object (e.g., character, object, person, animal) is included within the image. If no major object is included, the electronic device (101) can determine that it is a background image or a landscape image. The electronic device (101) can analyze the lock screen image based on either an on-device artificial intelligence model or a server artificial intelligence model.

[0128] According to one embodiment, the electronic device (101) may receive a notification or an application notification for the electronic device (101). The notification for the electronic device (101) may include, for example, a notification regarding battery information. The application notification may include, for example, a notification regarding a schedule or a notification regarding the receipt of a message. The types of notifications for the electronic device (101) and application notifications are merely examples and are not limited thereto. The electronic device (101) may generate a prompt requesting image creation by reflecting the notification content based on receiving a message from another user (e.g., girlfriend) terminal.

[0129] According to one embodiment, an electronic device (101) may provide a prompt as input to a generative artificial intelligence that includes information about a main object (e.g., character, object, person, animal, etc.), information about a remaining object (e.g., background, landscape), and information about notification content (e.g., type of application (SMS, messenger, email, calendar, reminder), sender, content of the notification, time of occurrence). For example, the electronic device (101) may generate at least a portion of the prompt based on at least temporarily stored rules. Alternatively, the electronic device (101) may generate at least a portion of the prompt using a generative artificial intelligence. For example, the electronic device (101) may provide the generative artificial intelligence with a request to generate a new prompt, including the main object, remaining objects, sender, and / or notification content, in the prompt. In one example, the electronic device (101) may search for external data (e.g., weather, social media information) and include at least a portion of text related to the searched data in the prompt. The electronic device (101) can generate a new image by inputting the generated prompt into a generative artificial intelligence. For example, the electronic device (101) can recognize a 'light-colored wooden floor' as an object for the lock screen set by the user. The electronic device (101) can recognize a 'kitten resembling a Korean stray cat with ochre stripes' (512) as a main object. The electronic device (101) can receive a notification about a message. The electronic device (101) can confirm that the message has been received from the girlfriend's terminal. The electronic device (101) can decide to express the action of confirming a notification (522) about a message received from the girlfriend, with the object for the 'light-colored wooden floor' as the background and the 'kitten resembling a Korean stray cat with ochre stripes' (512) as the main object.

[0130] In this case, the electronic device (101) may generate a prompt requesting the output of an image of a kitten resembling a Korean stray cat with ochre stripes on a light-colored wooden floor background checking a notification message received from a girlfriend. The background and object are merely examples and are not limited thereto. The type of application is also not limited to a message application. For example, the electronic device (101) may receive various application notifications including information about the weather, time, biometric data, or battery level. The electronic device (101) may generate an image related to the application notification by providing the received application notification as a prompt. The electronic device (101) may display time information (515).

[0131] In Figure 520, the electronic device (101) may provide a prompt as input to the generative artificial intelligence that includes information about a main object (e.g., character, object, person, animal, etc.), information about a remaining object (e.g., background, landscape), and information about notification content (e.g., type of application (SMS, messenger, email, calendar, reminder), sender, content of the notification, time of occurrence). The electronic device (101) may collect information about the main object (e.g., character, object, person, animal) and the remaining object (e.g., background, landscape) and provide it as input to the generative artificial intelligence. A character may refer to an object with a specific personality or characteristic. An object may refer to a thing with a specific function or attribute. Information about a person may include information about a person's face, posture, and clothing. An animal may refer to a specific species of animal or a pet (e.g., dog, cat, etc.). A background may refer to the surrounding environment to be used in an image. A landscape may include an image representing a specific place or situation. The electronic device (101) can display time information (525).

[0132] In one embodiment, the electronic device (101) can structure the collected information into a clear and systematic format. For example, the electronic device (101) can organize the data into JSON or XML format and convert it into a format easily understood by the generative AI. The electronic device (101) can input the structured data into the generative AI and request it to generate a new image.

[0133] An electronic device (101) can generate a new image using generative artificial intelligence. The electronic device (101) can display the generated image and set it as the lock screen. The generative artificial intelligence can generate an image by arranging a main object and other objects. The generative artificial intelligence can generate an image with a specific style or mood (e.g., warm or modern) based on information requested in a prompt. The generative artificial intelligence can perform post-processing, including color adjustment, resolution enhancement, and removal of unnecessary elements. The electronic device (101) can determine the final generated image as the lock screen and display a notification to the user.

[0134] In Figure 530, the electronic device (101) can receive multiple messages from another user's (e.g., girlfriend's) terminal within a threshold time. The electronic device (101) can provide a prompt as input to the generative artificial intelligence, which includes information about a main object (e.g., character, object, person, animal, etc.), information about the remaining objects (e.g., background, landscape), and information about the notification content (e.g., type of application (SMS, messenger, email, calendar, reminder), sender, notification content, occurrence time). The electronic device (101) can generate a new image using the generative artificial intelligence. The electronic device (101) can display the generated image. In Figure 520, a single message (522) is displayed, whereas in Figure 530, multiple messages (532) can be displayed. The electronic device (101) may display a set emotion (534) based on information about the notification content (e.g., type of application (SMS, messenger, email, calendar, reminder), sender, notification content, and occurrence time). For example, the electronic device (101) may display an image indicating a happy emotion based on the confirmation that the sender of the message is a 'girlfriend'. Alternatively, the electronic device (101) may display an image indicating a nervous emotion based on the confirmation that the sender of the message is a 'boss at work'. The electronic device (101) may determine an emotion corresponding to the sender based on a user input. If an emotion corresponding to the sender is not set, the electronic device (101) may not display the emotion image (534) or may display a neutral image. The electronic device (101) may display time information (535).

[0135] FIG. 6 illustrates the process of generating a prompt and an image in response to a notification in an electronic device according to one embodiment.

[0136] Figure 610 may represent a lock screen (or background) set by a user of an electronic device (e.g., the electronic device (101) of FIG. 1). According to one embodiment, the electronic device (101) may generate a prompt requesting that Figure 610 be analyzed and converted into text. The electronic device (101) may use the generated prompt to determine that Figure 610 is an "image of a woman tearing at her hair with an annoyed expression."

[0137] According to one embodiment, an electronic device (101) can analyze a lock screen set by a user and convert it into text. The electronic device (101) can analyze the lock screen set by the user using a processor (120) or a generative artificial intelligence. The electronic device (101) can generate a prompt requesting the analysis of the lock screen set by the user and conversion into text. The electronic device (101) can analyze the set lock screen by providing the generated prompt as input to the generative artificial intelligence. Based on the analysis results, the electronic device (101) can separate and recognize the background and objects of the set lock screen and distinguish the type of the lock screen.

[0138] According to one embodiment, an electronic device (101) can generate an object image similar to object features based on an image containing an object (e.g., a person) using generative artificial intelligence. The electronic device (101) can generate a personalized prompt capable of regenerating an object image by analyzing an object (e.g., a face) and a background within an image.

[0139] According to one embodiment, an electronic device (101) may determine that a person's face is recognized if it includes features such as eyes, nose, and mouth, which are major parts of a face within an image, using a face recognition algorithm (e.g., PCA, principal component analysis). The electronic device (101) may determine an image theme (e.g., portrait theme) optimized for the recognized object (e.g., person or face) based on object and background features (e.g., face features, color tone, background complexity) analyzed through image analysis, and generate an image.

[0140] According to one embodiment, the electronic device (101) may determine at least one image theme optimized for (or recommended for) the face of a person included in the image (610) based on the image analysis results, and generate a prompt that may be involved in generating an object image to which the image theme style is applied.

[0141] An electronic device (101) according to one embodiment may output a specified prompt (e.g., a template prompt) in relation to an image theme (or style). For example, an image theme may consist of a plurality of image themes (e.g., illustration, 3D character, vintage, watercolor, cartoon, etc.). For each image theme, the prompt for generating an object image suitable for each theme style may differ. For example, for a cartoon style image theme, a prompt to change an input object to a cartoon style may be required. For a watercolor style image theme, a prompt to change an input object to a watercolor style may be required. The prompt related to the image theme may be stored in the electronic device (101) in a pre-specified form. The electronic device (101) may display time information (615).

[0142] According to one embodiment, the electronic device (101) may display a notification (620) regarding the remaining battery level. For example, the electronic device (101) may display a notification indicating that the battery is low when the battery is below (or less than) a threshold. The electronic device (101) may analyze the content of the notification (620) regarding the remaining battery level. Additionally, the electronic device (101) may analyze the lock screen (or background screen) using a processor (e.g., the processor (120) of FIG. 1) or generative artificial intelligence. The lock screen may be set by the user or may be a screen generated by artificial intelligence.

[0143] According to one embodiment, the electronic device (101) may display a notification (620) based on whether the remaining battery level is below a specified level (e.g., 15%). The electronic device (101) may analyze the content of the notification (620) to determine whether it is positive or negative. For example, the electronic device (101) may determine that a notification indicating that the battery is fully charged is positive. Whether a specific notification is positive, negative, or neutral may be defined in advance during the development phase. Alternatively, whether a specific notification is positive, negative, or neutral may be determined by user input.

[0144] In one embodiment, the electronic device (101) may determine that a notification indicating that the battery level is below a specified level (e.g., 15%) is negative. The electronic device (101) may analyze the lock screen image to determine that a person is displayed. The electronic device (101) may combine the displayed person with the negative situation to generate a prompt such as, "Draw a person with a negative expression. The person is the same person as the person appearing in the image (610). Also, draw a battery so that we can understand that this is a battery charging situation." The electronic device (101) may provide the generated prompt as an input to the processor (120) or generative artificial intelligence to generate a new image (640). The electronic device (101) may display the generated image on the display. The electronic device (101) may use the new image (640) to provide the user with information indicating that the battery level is below a specified level and that charging is necessary. The electronic device (101) may display time information (645).

[0145] FIG. 7 illustrates the process of generating a prompt and an image in response to a notification in an electronic device according to one embodiment.

[0146] According to one embodiment, an electronic device (e.g., the electronic device (101) of FIG. 1) can analyze a lock screen set by a user and convert it into text. The electronic device (101) can analyze a lock screen set by a user using a processor (120) or generative artificial intelligence.

[0147] Figure 710 may represent a lock screen (or background image) set by a user. An electronic device (101) may analyze Figure 710 to determine the characteristics of a main object (e.g., a character) (712) and the remaining objects. According to one embodiment, the electronic device (101) may determine that the set lock screen image is an image of a background based on the analysis of the set lock screen image and whether the object is not displayed in the display area or the size of the area occupied by the object is less than a specified level. The electronic device (101) may determine that the set lock screen image is an image of an object based on whether the object is displayed in the display area and the size of the area occupied by the object is greater than or equal to a specified level.

[0148] According to one embodiment, the electronic device (101) can generate a prompt requesting a textual description of Figure 710 by analyzing Figure 710. The electronic device (101) can use the generated prompt to determine that the description of Figure 710 is "an image of a puppy character with spotted markings on a pink background sitting and looking to the left with a happy expression." The electronic device (101) can display time information (715).

[0149] In Figure 720, the electronic device (101) can receive multiple application notifications (722, 724, 726).

[0150] Application notifications may include, for example, notifications (722, 724) of an application related to e-commerce. The electronic device (101) may receive notifications from an application related to e-commerce indicating that delivery has been completed and / or that delivery is delayed. The types of applications and notification content are examples only and are not limited thereto.

[0151] Application notifications may include, for example, a message reception notification (726). The electronic device (101) may generate a prompt requesting image generation based on the notification content upon receiving a message from another user (e.g., a team leader) terminal.

[0152] According to one embodiment, an electronic device (101) can perform text analysis on an application notification and summarize it. The electronic device (101) can analyze the content of the application notification using a processor (120) or generative artificial intelligence. The electronic device (101) can analyze the application notification and generate a prompt requesting a description in text. For example, the electronic device (101) can perform text analysis on the application notification and summarize it as "There are 12 unread messages from the team leader."

[0153] According to one embodiment, the electronic device (101) may determine the display location of the notification and the form of the image to be generated differently based on importance upon receiving a plurality of application notifications. The importance of the notification may be determined in chronological order or may be determined by the user in advance. Alternatively, the electronic device (101) may determine the importance of the application notification based on the user's application usage time and usage frequency within a specific period (e.g., the last week). The electronic device (101) may set the importance of the application notification relatively higher as the application usage time is longer and the usage frequency is higher. Conversely, the electronic device (101) may set the importance of the application notification relatively lower as the application usage time is shorter and the usage frequency is lower.

[0154] According to one embodiment, when the number of received application notifications is multiple, the electronic device (101) may generate prompts for each application notification, and provide each generated prompt as input to the processor (120) or an artificial intelligence model (e.g., generative artificial intelligence) to generate a new image. The electronic device (101) may determine the importance of the application notification according to preset criteria, and may determine a different location on the display for the generated image based on the importance.

[0155] In one embodiment, the electronic device (101) may generate a prompt (730) based on the content and importance of multiple application notifications (720), the background of the lock screen image (710), and object analysis. For example, the electronic device (101) may determine that the background image of the lock screen image (710) is a pink screen and the main object is a spotted puppy character (712).

[0156] In one embodiment, the electronic device (101) may generate a prompt based on analysis of objects and text analysis of application notifications. For example, the electronic device (101) may generate a prompt requesting the display of an image of a spotted puppy character waiting for 12 unread messages from the team leader. The content of the prompt and the displayed image may vary depending on the results of the generative artificial intelligence.

[0157] According to one embodiment, the electronic device (101) may generate a prompt based on the content of a first application notification received first in time. The first application is an application related to e-commerce, and the first application notification may be, for example, a notification indicating that delivery has been completed. The electronic device (101) may generate a prompt such as, "Create an image of a spotted puppy character checking the delivered items on a pink background." The electronic device (101) may determine through natural language analysis that the delivery has been completed is a positive situation and may also include in the prompt a request to draw the character's expression in a positive way.

[0158] According to one embodiment, the electronic device (101) may provide a third user interface (e.g., the third user interface (930) of FIG. 9) that allows the user to select whether to generate an image in a positive direction or in a negative direction when an application notification is received. The electronic device (101) may determine whether to generate an image in a positive direction or in a negative direction on an application-by-application basis based on user input to the third user interface (930). For example, the electronic device (101) may include content in the prompt requesting to make a positive facial expression or draw a background with a bright atmosphere based on the receipt of a notification from an application determined to generate an image positively.

[0159] On the other hand, the electronic device (101) may include content requesting a negative expression or a dark background in the prompt based on receiving a notification from an application determined to generate a negative image. Positive and negative are just examples of emotions, and emotions are not limited to these. For example, various emotions such as anger, urgency, joy, and surprise may be displayed in the third user interface (930).

[0160] According to one embodiment, the electronic device (101) may display an output image corresponding to a first application notification of the highest relative importance in a first area (741) that has the largest area occupied by the display. Additionally, the electronic device (101) may emphasize importance by making the image quality of the image displayed in the first area (741) relatively higher compared to the image displayed in other areas, and making the brightness the highest compared to other areas. The electronic device (101) may make the clarity of the image displayed in the first area (741) relatively higher compared to the image displayed in other areas.

[0161] According to one embodiment, the electronic device (101) may generate a prompt based on the content of a second application notification received later in time than the first application notification. The second application is an application related to e-commerce, and the second application notification may be, for example, a notification indicating that the delivery of a blouse has been delayed. The electronic device (101) may generate a prompt such as, "Create an image of a spotted puppy character holding a blouse and delivering it on a pink background." The electronic device (101) may determine through natural language analysis that the content regarding the delivery delay is a negative situation and may also include in the prompt a request to draw the character's expression negatively. The electronic device (101) may generate an output image corresponding to the second application notification based on the generated prompt.

[0162] According to one embodiment, the electronic device (101) may display an output image corresponding to a second application notification, which has relatively lower importance than a first application notification, in a second area (742). The second area (742) may occupy a relatively smaller size on the display than the first area (741). Additionally, the electronic device (101) may lower the quality of the image displayed in the second area (742) by comparing it to the image displayed in the first area (741). The electronic device (101) may set the brightness of the image displayed in the second area (742) to be relatively lower compared to the brightness of the image displayed in the first area (741). The electronic device (101) may set the clarity of the image displayed in the second area (742) to be relatively lower compared to the clarity of the image displayed in the first area (741).

[0163] According to one embodiment, the electronic device (101) may generate a prompt based on the content of a third application notification received later in time than the second application notification. The third application is an application related to communication, and the third application notification may be a notification indicating that a message has been received from another user (e.g., Team Leader) terminal. The electronic device (101) may generate a prompt such as, "Create an image where a character corresponding to the Team Leader is displayed on a pink background screen along with the number of received messages." The character corresponding to the "Team Leader" may be determined by the user in advance. Alternatively, if the character corresponding to the "Team Leader" is not determined, the electronic device (101) may generate and display any character (e.g., person or question mark). The character corresponding to the "Team Leader" is merely an example and may vary depending on the settings, and is not limited thereto. The electronic device (101) may pre-set a character corresponding to the recipient (e.g., girlfriend, Team Leader) based on user input. The recipient is not limited to this example. The electronic device (101) may generate and display a random character (e.g., a person or a question mark) if a preset character is not identified.

[0164] According to one embodiment, the electronic device (101) can determine the character's facial expression based on the settings for another user (e.g., a team leader) who sent the message. For example, the other user (e.g., a team leader) may be set to have a negative emotion. When the electronic device (101) receives a notification from another user (e.g., a team leader) who is set to have a negative emotion, it may include a prompt requesting that the character's facial expression be drawn negatively. The electronic device (101) can generate an output image corresponding to a third application notification based on the generated prompt.

[0165] According to one embodiment, the electronic device (101) can determine a corresponding character based on information about the subject that sent the application notification. The electronic device (101) can determine the facial expression, posture, and movement of the corresponding character based on the content of the application notification. The electronic device (101) can generate a prompt based on the information about the determined character and the information about the character's facial expression, posture, and movement, and can generate a new image by providing the generated prompt as input to a processor (120) or an artificial intelligence model.

[0166] According to one embodiment, the electronic device (101) may display an output image corresponding to a third application notification, which has relatively lower importance than a second application notification, in a third area (743). The third area (743) may occupy a relatively smaller size on the display than the second area (742). Additionally, the electronic device (101) may lower the quality of the image displayed in the third area (743) by comparing it to the image displayed in the second area (742). The electronic device (101) may set the brightness of the image displayed in the third area (743) to be relatively lower compared to the brightness of the image displayed in the second area (742). The electronic device (101) may set the clarity of the image displayed in the third area (743) to be relatively lower compared to the clarity of the image displayed in the second area (742).

[0167] According to one embodiment, the electronic device (101) can display an output image corresponding to an application notification set to have relatively high importance in a first area (741) that is relatively large in size on the display. The electronic device (101) can display an output image corresponding to an application notification set to have relatively low importance in a second area (742) that is relatively small in size on the display.

[0168] According to one embodiment, the electronic device (101) may determine an area on the display where no image should be displayed or a background should be displayed instead of an object in order to display time information or notification information. The electronic device (101) may display time information and notification information in a space (745) between the first area (741), the second area (742), and the third area (743). The electronic device (101) may generate a prompt to determine that the background should be displayed instead of an object in the space (745) between the first area (741), the second area (742), and the third area (743) and that the complexity of the image should be low. The complexity may be considered high when a 3D object is displayed, and low when a background is displayed. For example, the electronic device (101) may generate a prompt requesting that 'a background should be drawn instead of an object in an area from width 00 to width XX and from height 00 to height XX'. The electronic device (101) can provide the user with a summary of the application notification by displaying an interface (747) that displays the application notification in a condensed form.

[0169] FIG. 8 illustrates the process of generating a prompt and an image in response to a notification in an electronic device according to one embodiment.

[0170] Figure 810 may refer to a lock screen (or background screen) image set by a user of an electronic device (e.g., the electronic device (101) of Figure 1). The electronic device (101) may analyze Figure 810 using a processor (e.g., the processor (120) of Figure 1) or generative artificial intelligence. The electronic device (101) may generate a prompt requesting that Figure 810 be analyzed and converted into text. Using the generated prompt, the electronic device (101) may determine the description of Figure 810 as 'a landscape image of a cold winter mountain across a lake on a dark night.'

[0171] In one embodiment, the electronic device (101) may receive a notification (820). The notification (820) may include, for example, a weather-related notification (822) or a notification (824) regarding a message received from a specific application (e.g., an aviation application). The weather-related notification (822) may, for example, include news of sudden snowfall. The message (824) received from the aviation application may include information regarding the departure time of a reserved flight and guidance on the check-in process. The electronic device (101) may analyze the message received from the aviation application to determine an object to add to the prompt as an airplane.

[0172] According to one embodiment, the electronic device (101) may generate a prompt (830) based on the analysis results of Figure 810 and the contents of the received notification (820). The electronic device (101) may generate a prompt with content that generates a landscape image of a cold winter mountain across a lake on a dark night as a background. The electronic device (101) may generate a prompt with content that generates a landscape image of snow falling and an airplane flying over as a background, based on the current time (e.g., 5:55 AM) and a weather notification. The electronic device (101) may generate a prompt with content that generates a landscape image of an airplane flying over as a background, based on the current time (e.g., 5:55 AM). The electronic device (101) may generate a prompt with content that generates a landscape image of an airplane flying over as a background, based on the notification (824) of an aviation application. Since early morning and dark night are difficult to reconcile, the electronic device (101) may determine the content of the prompt based on the current time. The electronic device (101) may generate a prompt that requests to output a 'landscape image of a cold winter mountain with snow falling and an airplane flying over as a background, in the early morning.'

[0173] According to one embodiment, the electronic device (101) can generate a new image (840) by providing the generated prompt (830) as input to a processor (120) or a generative artificial intelligence. The electronic device (101) can display the generated image (840) on a display. The electronic device (101) can provide information to the user through the generated image (840) indicating that time information (844) that it is early morning, weather information (822) that it is snowing, and notifications (824) from an aviation application have been received. The electronic device (101) can provide the user with general information about the application notification by displaying an interface (847) that summarizes the application notification.

[0174]

[0175] FIG. 9 illustrates a user interface for controlling elements necessary for image generation in an electronic device according to one embodiment.

[0176] According to one embodiment, in FIG. 910, an electronic device (e.g., the electronic device (101) of FIG. 1) may provide a first user interface (912) that can select elements to be reflected on the background screen. For example, the electronic device (101) may provide an interface to determine whether to reflect each of the elements, including time, biometric information, application (app) notifications, and system notifications, on the background screen. The biometric information may include, for example, information regarding heart rate or stress index. The system notifications may include, for example, notifications regarding battery level. The application (app) notifications may include the messaging application, e-commerce-related application, and aviation-related application described in FIG. 5 through 8. The types of biometric information, system notifications, and application notifications are merely examples and are not limited thereto.

[0177] In Figure 920, the electronic device (101) may provide a second user interface (922) that allows the user to select a notification to be reflected in the creation of a lock screen (or background screen). The applications (apps) may include the messaging applications, e-commerce-related applications, and aviation-related applications described in Figures 5 to 8 . The types of applications are merely examples and are not limited thereto. The electronic device (101) may determine an application to display an application notification based on a user input to the second user interface (922). If a notification is received from an application determined to display an application notification, the electronic device (101) may reflect the content of the notification in a prompt to create a new lock screen image. If a notification is received from an application determined not to display an application notification, the electronic device (101) may not reflect the content of the notification in a prompt.

[0178] According to one embodiment, the electronic device (101) may provide a third user interface (930) that allows the user to select whether to generate an image in a positive direction or in a negative direction when an application notification is received. The electronic device (101) may determine whether to generate an image in a positive direction or in a negative direction for each application based on user input to the third user interface (930). For example, the electronic device (101) may include content in the prompt requesting a positive expression (932) or a background with a bright atmosphere based on the receipt of a notification from an application determined to generate an image in a positive direction. On the other hand, the electronic device (101) may include content in the prompt requesting a negative expression (934) or a background with a dark atmosphere based on the receipt of a notification from an application determined to generate an image in a negative direction. Positive and negative are merely examples of emotions, and emotions are not limited to these. For example, various emotions such as anger, urgency, joy, and surprise can be displayed on the third user interface (930).

[0179] FIG. 10 is a flowchart illustrating a method for creating a lock screen of an electronic device according to one embodiment.

[0180] The operations described through FIG. 10 may be implemented based on instructions that can be stored in a computer recording medium or memory (e.g., memory (130) of FIG. 1). The illustrated method may be executed by an electronic device (e.g., electronic device (101) of FIG. 1) described above through FIG. 1 to 9, and the technical features described above will be omitted below. The order of each operation in FIG. 10 may be changed, some operations may be omitted, and some operations may be performed simultaneously.

[0181] In operation 1010, the electronic device (101) can analyze a lock screen image to obtain information about the background and information about the presence of objects. According to one embodiment, the electronic device (101) can analyze a lock screen set by a user and convert it into text. The electronic device (101) can analyze a lock screen set by a user using a processor (120) or generative artificial intelligence. According to one embodiment, the electronic device (101) can analyze Figure 710 and generate a prompt requesting a description in text.

[0182] According to one embodiment, the electronic device (101) can separate and recognize the background and objects of a set lock screen based on the analysis content and distinguish the type of the lock screen. The type of the lock screen can be divided into whether it is an image containing a person, an image of a landscape, or an image containing an object. The object may include, for example, at least one of a person, an animal, a plant, a character, or an object. The type of the lock screen is merely an example and is not limited thereto. The type of object is also merely an example and is not limited thereto.

[0183] In operation 1020, the electronic device (101) may receive a notification and generate a prompt requesting the creation of an image. The electronic device (101) may determine a priority based on importance for at least one of the information, such as time, application notifications, or biometric information, and generate a prompt according to the priority.

[0184] According to one embodiment, importance may be determined in the order of time when information is generated, or it may be determined by the user in advance. Alternatively, the electronic device (101) may determine the importance of application notifications based on the user's application usage time and usage frequency within a specific period (e.g., the last week). The electronic device (101) may set the importance of application notifications relatively high as the application usage time is long and usage frequency is high. On the other hand, the electronic device (101) may set the importance of application notifications relatively low as the application usage time is short and usage frequency is low.

[0185] According to one embodiment, the electronic device (101) may receive application notifications. The application notifications may include, for example, a message reception notification. Based on receiving a message from another user (e.g., girlfriend) terminal, the electronic device (101) may generate a prompt requesting image creation by reflecting the content of the notification.

[0186] According to one embodiment, an electronic device (101) may provide a prompt to a generative artificial intelligence as input, which includes information about a main object (e.g., character, object, person, animal, etc.), information about a remaining object (e.g., background, landscape), and information about the notification content (e.g., type of application (SMS, messenger, email, calendar, reminder), sender, content of the notification, time of occurrence).

[0187] In operation 1030, the electronic device (101) can generate a new image by providing the generated prompt as input to an artificial intelligence model. The artificial intelligence model may include a generative artificial intelligence. According to one embodiment, the generative artificial intelligence is a type of artificial intelligence model and may be placed inside the electronic device (101) or on an external server. According to one embodiment, the processor (120) may provide the generative artificial intelligence to execute an image generation process based on the generated instructions. According to one embodiment, the generated instructions may be provided to the generative artificial intelligence on the device and / or the generative artificial intelligence on the server.

[0188] In operation 1040, the electronic device (101) can set the generated image as a lock screen image.

[0189] According to one embodiment, the electronic device (101) may display an image corresponding to a first application notification of the highest relative importance in a first area (e.g., the first area (741) of FIG. 7) that has the largest area occupying the display. Additionally, the electronic device (101) may emphasize the importance by making the image quality of the image displayed in the first area (741) relatively higher compared to the image displayed in other areas, and making the brightness the largest compared to other areas. The electronic device (101) may make the clarity of the image displayed in the first area (741) relatively higher compared to the image displayed in other areas.

[0190] According to one embodiment, the electronic device (101) may display an output image corresponding to a second application notification, which has relatively lower importance than a first application notification, in a second area (e.g., the second area (742) of FIG. 7). The second area (742) may occupy a relatively smaller size on the display than the first area (741). Additionally, the electronic device (101) may lower the quality of the image displayed in the second area (742) by comparing it to the image displayed in the first area (741). The electronic device (101) may set the brightness of the image displayed in the second area (742) to be relatively lower compared to the brightness of the image displayed in the first area (741). The electronic device (101) may set the clarity of the image displayed in the second area (742) to be relatively lower compared to the clarity of the image displayed in the first area (741).

[0191] According to one embodiment, the electronic device (101) may display an output image corresponding to a third application notification, which is relatively less important than a second application notification, in a third area (e.g., the third area (743) of FIG. 7). The third area (743) may occupy a relatively smaller size on the display than the second area (742). Additionally, the electronic device (101) may lower the quality of the image displayed in the third area (743) by comparing it to the image displayed in the second area (742). The electronic device (101) may set the brightness of the image displayed in the third area (743) to be relatively lower compared to the brightness of the image displayed in the second area (742). The electronic device (101) may set the clarity of the image displayed in the third area (743) to be relatively lower compared to the clarity of the image displayed in the second area (742).

[0192] According to one embodiment, the electronic device (101) can set the image displayed in the first area (741) to have the highest image quality and brightness relative to the image displayed in the first area (741). The electronic device (101) can set the image displayed in the second area (742) to have lower image quality and brightness relative to the image displayed in the first area (741). The electronic device (101) can control the image displayed in the third area (743) to have lower image quality and brightness relative to the image displayed in the second area (742).

[0193] According to one embodiment, when the electronic device (101) receives notifications from a first application, a second application, and a third application, the electronic device (101) may determine the importance of the first application, the second application, and the third application based on preset criteria. The electronic device (101) may display an output image corresponding to the notification of the first application having a relatively highest importance on the display. The electronic device (101) may determine not to display output images corresponding to the notifications of the second application and the third application on the display.

[0194] FIG. 11 is a flowchart illustrating a method for creating a lock screen of an electronic device according to one embodiment.

[0195] The operations described through FIG. 11 can be implemented based on instructions that can be stored in a computer recording medium or memory (e.g., memory (130) of FIG. 1). The illustrated method can be executed by the electronic device described above through FIGS. 1 to 9 (e.g., electronic device (101) of FIG. 1), and the technical features described above will be omitted below. The order of each operation of FIG. 11 can be changed, some operations can be omitted, and some operations can be performed simultaneously.

[0196] In operation 1110, the electronic device (101) can check whether an object exists in the set lock screen image. The object may include, for example, any one of a person, an animal, or a character. The types of objects are merely examples and are not limited thereto.

[0197] According to one embodiment, the electronic device (101) can analyze a lock screen set by a user and convert it into text format. The electronic device (101) can analyze the lock screen set by the user using the processor (120) or generative artificial intelligence. The electronic device (101) can generate a prompt requesting that the lock screen set by the user be analyzed and converted into text. The electronic device (101) can analyze the set lock screen by providing the generated prompt as input to the generative artificial intelligence. Based on the analysis results, the electronic device (101) can separate and recognize the background and objects of the set lock screen and distinguish the type of the lock screen. The type of the lock screen can be divided into an image including a person, an image of a landscape, or an image including an object.

[0198] In operation 1114, the electronic device (101) may determine that no object exists. Based on the determination that no object exists in the image, the electronic device (101) may generate a prompt in operation 1140 that does not contain the main object.

[0199] For example, the electronic device (101) can analyze Figure 810 of FIG. 8 and generate a prompt requesting a description in text. The electronic device (101) can use the generated prompt to determine the description of Figure 810 as ‘a landscape image of a cold winter mountain beyond a lake on a dark night.’

[0200] According to one embodiment, the electronic device (101) may receive a notification. The notification may include, for example, a weather-related notification. The weather-related notification may include, for example, news that it is snowing suddenly. The electronic device (101) may generate a prompt based on the analysis of Figure 810 and the content of the received notification. In this case, the prompt may not include content regarding the main object. The electronic device (101) may generate a prompt with content created against a landscape image depicting a dark night and cold winter mountains beyond a lake. The electronic device (101) may generate a prompt with a background of snow falling and early morning based on the current time (e.g., 5:55 AM) and the weather notification. Since early morning and dark night are difficult to coexist, the electronic device (101) may determine the content of the prompt based on the current time. The electronic device (101) can generate a prompt requesting to output a landscape image of a cold winter mountain beyond a lake with snow falling in the early morning background.

[0201] According to one embodiment, the electronic device (101) can determine whether to display the sun on the background screen based on weather information and can generate a prompt to not display the sun when it is cloudy, raining, or snowing. The electronic device (101) can generate a prompt to change the position where the sun is rising on the background screen based on time information. The electronic device (101) can provide the generated prompt as input to a processor (120) or an artificial intelligence model to generate a new image.

[0202] According to one embodiment, the electronic device (101) can adjust the brightness change of the background image based on a time zone.

[0203] In one embodiment, the electronic device (101) may determine the pose of an object to be included in a prompt based on the mode of the electronic device (101). For example, the electronic device (101) may generate a prompt indicating that the pose of the object is sitting when the electronic device (101) is in a folded state. The state of the electronic device (101) and the pose of the object are merely examples and are not limited thereto.

[0204] In operation 1120, the electronic device (101) can determine whether the number of objects present is one or multiple based on the presence of an object. In operation 1122, the electronic device (101) can determine an object in the image as a primary object based on the presence of one object, and can generate a prompt including the primary object in operation 1130. In operation 1124, the electronic device (101) can determine an object occupying the largest area in the image as the primary object based on the presence of multiple objects in the image. Based on the determination of the primary object, the electronic device (101) can generate a prompt including the primary object in operation 1130.

[0205] In operation 1150, the electronic device (101) can generate a new image by inputting a prompt into an artificial intelligence model. The electronic device (101) can generate a new image by providing the generated prompt to a processor (120) or a generative artificial intelligence.

[0206] According to one embodiment, the electronic device (101) may generate a prompt requesting the generation of a relaxing image capable of lowering the heart rate based on receiving a notification indicating that the user's heart rate exceeds a specified level. The electronic device (101) may generate a prompt requesting the generation of a relaxing image capable of lowering the stress index based on receiving a notification indicating that the user's stress index exceeds a specified level, and may provide the generated prompt as input to the processor (120) or an artificial intelligence model to generate a new image.

[0207] The embodiments of this document disclosed in this specification and drawings are merely specific examples to easily explain the technical contents according to the embodiments of this document and to help understand the embodiments of this document, and are not intended to limit the scope of the embodiments of this document. Therefore, the scope of one embodiment of this document should be interpreted to include all changes or modified forms derived based on the technical idea of ​​one embodiment of this document, in addition to the embodiments disclosed herein.

Claims

1. In electronic devices, Memory that stores instructions and includes one or more storage media; It includes at least one processor comprising processing circuitry, and When the above instructions are executed individually or collectively by the at least one processor, the electronic device Using the above processor or artificial intelligence model, convert a lock screen image set on the electronic device into text form, and Obtain information about a main object including any one of a person, character, object, or animal, and information about the remaining objects, and Based on receiving an application notification, generate a prompt requesting the creation of an image using the received notification content, information about the main object, and information about the remaining objects, and Generate a new image by providing the generated prompt as input to the processor or the artificial intelligence model, and An electronic device that controls the setting of the above-mentioned generated image as a lock screen image.

2. In paragraph 1, When the above instructions are executed individually or collectively by the at least one processor, the electronic device Based on the number of received application notifications being multiple, prompts are generated for each application notification, and Each generated prompt is provided as input to the processor or the artificial intelligence model to generate a new image for each, and The importance of an application notification is determined based on at least one of the time the application notification arrived, a pre-set standard, or the frequency of application usage. An electronic device that controls the position where an image generated based on importance is displayed on a display.

3. In paragraph 2, When the above instructions are executed individually or collectively by the at least one processor, the electronic device An image corresponding to an application notification set to have relatively high importance is displayed in a first area of ​​relatively large size on the display, and An electronic device that controls an image corresponding to an application notification set to have relatively low importance to be displayed in a second area of ​​relatively small size on the display.

4. In paragraph 2, When the above instructions are executed individually or collectively by the at least one processor, the electronic device In a situation where notifications are received from the first application, the second application, and the third application, the importance of the first application, the second application, and the third application is determined according to pre-set criteria, and The output image corresponding to the first application notification, which has the highest relative importance, is displayed in the first area that occupies the largest area on the display, and An output image corresponding to a second application notification of relatively medium importance is displayed in a second area on the display where the size of the area occupied is smaller than the first area, and An electronic device that controls the output image corresponding to the third application notification, which has the lowest relative importance, to be displayed in a third area on the display where the size of the area occupied is relatively smaller than that of the second area.

5. In paragraph 4, When the above instructions are executed individually or collectively by the at least one processor, the electronic device The image displayed in the first area above is set so that the image quality and brightness are relatively the highest, and The image displayed in the second area is set to have relatively lower quality and brightness than the image displayed in the first area. An electronic device that controls the image displayed in the third area to have relatively lower image quality and brightness than the image displayed in the second area.

6. In Paragraph 2, When the above instructions are executed individually or collectively by the at least one processor, the electronic device In a situation where notifications are received from the first application, the second application, and the third application, the importance of the first application, the second application, and the third application is determined according to pre-set criteria, and Displaying an output image corresponding to a notification of the first application having the highest relative importance on the display; An electronic device that controls output images corresponding to notifications of the second application and the third application not to be displayed on the display.

7. In paragraph 1, The above application notification includes at least one of time information, weather information, biometric information, or battery level information. When the above instructions are executed individually or collectively by the at least one processor, the electronic device Based on the above weather information, determine whether to display the sun on the background screen, and generate a prompt to not display the sun if it is cloudy, raining, or snowing, and Generate a prompt to change the position of the sun on the background screen based on the above time information, An electronic device that controls the generation of a new image by providing the generated prompt as input to the processor or the artificial intelligence model.

8. In paragraph 1, The above application notification includes at least one of time information, weather information, biometric information, or battery level information. When the above instructions are executed individually or collectively by the at least one processor, the electronic device Based on receiving a notification indicating that the user's heart rate of the electronic device has exceeded a specified level, a prompt is generated requesting the creation of a relaxing image that can lower the heart rate, and Based on receiving a notification indicating that the user's stress level of the electronic device has exceeded a specified level, a prompt is generated requesting the creation of a relaxing image that can lower the stress level, and An electronic device that controls the generation of a new image by providing the generated prompt as input to the processor or the artificial intelligence model.

9. In paragraph 1, When the above instructions are executed individually or collectively by the at least one processor, the electronic device Determine the corresponding character based on information about the subject who sent the above application notification, Based on the content of the above application notification, the corresponding character's facial expression, posture, and movement are determined. Generate prompts based on information about the determined character and information about the character's expressions, posture, and movements. An electronic device that controls the generation of a new image by providing the generated prompt as input to the processor or the artificial intelligence model.

10. In paragraph 1, When the above instructions are executed individually or collectively by the at least one processor, the electronic device As a result of analyzing the lock screen image set on the electronic device, it is determined that the set lock screen image is an image of the background based on the fact that the size or ratio of the area occupied by the main object in the display area is less than a specified level. Control is established to determine that the set lock screen image is an image of the object based on the fact that the size of the area occupied by the main object in the display area is greater than or equal to a specified level, and The above main object is an electronic device comprising at least one of a person, a character, an object, and an animal.

11. In the method of operating an electronic device, An operation of converting a lock screen image set on the electronic device into a text form using a processor or an artificial intelligence model, and obtaining information about a main object including any one of a person, character, object, and animal, and information about the remaining objects; An action of generating a prompt requesting the creation of an image using the received notification content, information about the main object, and information about the remaining objects, based on receiving an application notification; The operation of generating a new image by providing the generated prompt as input to the processor or the artificial intelligence model; and A method including the action of setting the generated image as a lock screen image.

12. In paragraph 11, An action of generating prompts for each application notification when there are multiple received application notifications; An operation of generating a new image for each generated prompt by providing it as input to the processor or the artificial intelligence model, respectively; An action that determines the importance of application notifications according to pre-set criteria; and A method further comprising an action that determines the position where an image generated based on importance is displayed on a display differently.

13. In paragraph 12, An operation of displaying an image corresponding to an application notification set to have relatively high importance in a first area of ​​relatively large size on the display; and A method further comprising the action of displaying an image corresponding to an application notification set to have relatively low importance in a second area of ​​relatively small size on the display.

14. In paragraph 12, An operation to determine the importance of the first application, the second application, and the third application based on preset criteria in a situation where notifications are received from the first application, the second application, and the third application; An operation to display an output image corresponding to the first application notification, which has the highest relative importance, in the first area of ​​the display that has the largest area occupied by the display; An operation of displaying an output image corresponding to a second application notification of relatively medium importance in a second area on the display where the size of the area occupied is smaller than the first area; and A method further comprising the operation of displaying an output image corresponding to a third application notification of the lowest relative importance in a third area on the display in which the size of the area occupied is relatively smaller than that of the second area.

15. In Paragraph 14, The image displayed in the first area above is set so that the image quality and brightness are relatively the highest, and The image displayed in the second area is set to have relatively lower image quality and brightness than the image displayed in the first area, and A method further comprising a control operation to control the image displayed in the third area to have relatively lower image quality and brightness than the image displayed in the second area.

Images