Method and apparatus for displaying image within frame in area corresponding to graphic object

Abstract

This electronic device may comprise: a display; at least one processor comprising processing circuitry; and a memory including one or more storage media in which instructions are stored, wherein, when executed by the at least one processor, the instructions can instruct the electronic device to: acquire a request for generating a graphic object for displaying a plurality of images; on the basis of an object detected in the corresponding image, determine, for respective images, a frame shape corresponding to a corresponding image; determine a display order of the plurality of images on the basis of a plurality of frame shapes corresponding to the plurality of images; determine, for the respective images, at least a portion of the corresponding image as a partial image on the basis of an object area in which the object is detected in the corresponding image and the frame shape corresponding to the corresponding image; and, in an area corresponding to the graphic object on a screen of the display, display, within a frame of a frame shape corresponding to a target image from among the plurality of images, a partial image of the target image for which the determined display order has been reached.

Description

Method and device for displaying an image through a frame in an area corresponding to a graphic object

[0001] A technique for displaying an image through a frame in an area corresponding to a graphic object is disclosed below.

[0002] The home screen can refer to user interface elements designed to conveniently provide users with various information. Widgets can refer to graphic objects on the home screen that display real-time information or help users quickly access specific app functions.

[0003] The information described above may be provided as related art for the purpose of aiding understanding of this document. None of the foregoing is to be claimed as prior art related to this document, nor is it to be used to determine prior art.

[0004] An electronic device according to one embodiment comprises: a display; at least one processor including a processing circuit; and a memory including one or more storage media for storing instructions. When the instructions are executed by the at least one processor, the electronic device may obtain a request to create a graphic object for displaying a plurality of images, for each of the plurality of images, determine a frame shape corresponding to the image based on an object detected in the image, determine a display order of the plurality of images based on the frame shapes corresponding to the plurality of images, and for each of the plurality of images, determine at least a part of the image as a partial image based on an object area where an object is detected in the image and a frame shape corresponding to the image, and display the partial image of the target image where the determined display order of the plurality of images has been reached through a frame of the frame shape corresponding to the target image in an area of ​​the screen of the display corresponding to the graphic object.

[0005] A method performed by an electronic device according to one embodiment may include: obtaining a request for the creation of a graphic object for displaying a plurality of images; for each of the plurality of images, determining a frame shape corresponding to the image based on an object detected in the image; determining the display order of the plurality of images based on the frame shapes corresponding to the plurality of images; for each of the plurality of images, determining at least a part of the image as a partial image based on an object region where an object is detected in the image and a frame shape corresponding to the image; and displaying the partial image of the target image, where the determined display order of the plurality of images has been reached, through a frame of the frame shape corresponding to the target image in an area corresponding to the graphic object on the screen of the display of the electronic device.

[0006] FIG. 1 is a block diagram illustrating an exemplary configuration of an electronic device according to various embodiments.

[0007] FIG. 2 is a diagram illustrating an example of an operation in which an electronic device according to various embodiments places a graphic object for displaying an image on a home screen.

[0008] FIG. 3 is a diagram illustrating an example of an operation in which an electronic device according to various embodiments changes the frame of a graphic object for displaying an image.

[0009] FIG. 4 is a drawing illustrating an example of a screen for changing the settings of a graphic object for displaying an image in an electronic device according to various embodiments.

[0010] FIG. 5 is a flowchart illustrating an example of a method for an electronic device according to various embodiments to generate a graphic object for displaying a plurality of images.

[0011] FIG. 6 is a diagram illustrating an example of an operation in which an electronic device according to various embodiments determines a frame shape corresponding to an image from a set of candidate frame shapes.

[0012] FIG. 7 is a diagram illustrating an example of an operation in which an electronic device according to various embodiments determines the display order of a plurality of images.

[0013] FIG. 8 is a diagram illustrating an example of an operation in which an electronic device according to various embodiments determines a partial image including at least a portion of an area outside the frame.

[0014] FIG. 9 is a diagram illustrating an example of an operation in which an electronic device according to various embodiments generates a supplementary image to supplement at least a portion of the internal area of ​​a frame.

[0015] FIG. 10 is a drawing illustrating an example of an operation to change the frame shape when the aspect ratio of an area corresponding to a graphic object is changed in an electronic device according to various embodiments.

[0016] FIG. 11 is a diagram illustrating an example of an operation in which a plurality of graphic objects are displayed on a home screen in an electronic device according to various embodiments, at least partially overlapping.

[0017] FIG. 12 is a drawing illustrating an example of an operation in which an electronic device according to various embodiments displays a home screen including a gridless page.

[0018] FIG. 13a is a drawing illustrating an example of an operation in which an electronic device according to various embodiments adds a shadow object to a graphic object.

[0019] FIG. 13b is a diagram illustrating an example of an operation in which an electronic device according to various embodiments adds a boundary object to a graphic object.

[0020] FIG. 13c is a drawing illustrating an example of an operation in which an electronic device according to various embodiments changes the visual properties of a graphic object.

[0021] FIG. 14 is a drawing illustrating examples of frame shapes according to changes in the size of graphic objects according to various embodiments.

[0022] Hereinafter, embodiments will be described in detail with reference to the attached drawings. In the description with reference to the attached drawings, identical components are given the same reference numeral regardless of the drawing number, and redundant descriptions thereof will be omitted.

[0023] FIG. 1 is a block diagram illustrating an exemplary configuration of an electronic device according to various embodiments.

[0024] FIG. 1 is a block diagram of an electronic device (101) in a network environment (100) according to various embodiments. Referring to FIG. 1, in the network environment (100), the electronic device (101) may communicate with an electronic device (102) through a first network (198) (e.g., a short-range wireless communication network) or may communicate with at least one of an electronic device (104) or a server (108) through a second network (199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (101) may communicate with the electronic device (104) through a server (108).

[0025] According to one embodiment, the electronic device (101) may include a processor (120), memory (130), input module (150), sound output module (155), display (160), audio module (170), sensor (176), interface (177), connection terminal (178), haptic module (179), camera (180), power management module (188), battery (189), communication module (190), subscriber identification module (196), or antenna module (197). In some embodiments, at least one of these components (e.g., connection terminal (178)) may be omitted from the electronic device (101), or one or more other components may be added. In some embodiments, some of these components (e.g., sensor (176), camera (180), or antenna module (197)) may be integrated into a single component (e.g., display (160)).

[0026] The processor (120) may be implemented as one or more IC (integrated circuit (or circuitry)) chips and may perform various data processing operations. The processor (120) may include at least one electrical circuit and may process instructions (or programs (140), data, etc.) stored in memory (130) individually or collectively in a distributed manner. The processor (120) may include a processor assembly comprising one or more processing circuits. The processor (120) may include any processing circuit that is operative to control the performance and operation of one or more components of the electronic device (101) (e.g., memory (130), display (160), camera (180), communication circuit, and / or sensor (176)).

[0027] The processor (120) can control at least one other component (e.g., hardware or software component) of the electronic device (101) connected to the processor (120) by executing software (e.g., program (140)), for example, and can perform various data processing or operations. According to one embodiment, as at least part of the data processing or operations, the processor (120) can store commands or data received from other components (e.g., sensor (176) or communication module (190)) in volatile memory (132), process the commands or data stored in volatile memory (132), and store the resulting data in non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., central processing unit or application processor) or an auxiliary processor (123) that can operate independently or together with it (e.g., graphics processing unit, neural processing unit (NPU), image signal processor, sensor hub processor, or communication processor). For example, if the electronic device (101) includes a main processor (121) and an auxiliary processor (123), the auxiliary processor (123) may be configured to use less power than the main processor (121) or to be specialized for a designated function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as part thereof.

[0028] The auxiliary processor (123) may control at least some of the functions or states associated with at least one component of the electronic device (101) (e.g., display (160), sensor (176), or communication module (190)) on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. According to one embodiment, the auxiliary processor (123) (e.g., image signal processor or communication processor) may be implemented as part of another functionally related component (e.g., camera (180) or communication module (190)). According to one embodiment, the auxiliary processor (123) (e.g., neural network processing unit) may include a hardware structure specialized for processing an artificial intelligence model. The artificial intelligence model may be generated through machine learning. Such learning may be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or through a separate server (e.g., server (108)). The learning algorithm may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model may include a plurality of artificial neural network layers.An artificial neural network may be a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to the hardware structure, the artificial intelligence model may include a software structure, either additionally or substantially.

[0029] The memory (130) can store various data used by at least one component of the electronic device (101) (e.g., processor (120) or sensor (176)). The data may include, for example, input data or output data for software (e.g., program (140)) and related instructions. The memory (130) may include volatile memory (132) or non-volatile memory (134).

[0030] The program (140) may be stored as software in memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0031] The input module (150) can receive commands or data to be used for a component of the electronic device (101) (e.g., processor (120)) from outside the electronic device (101) (e.g., user). The input module (150) may include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0032] The sound output module (155) can output a sound signal to the outside of the electronic device (101). The sound output module (155) may include, for example, a speaker or a receiver. The speaker may be used for general purposes, such as multimedia playback or recording playback. The receiver may be used to receive incoming calls. According to one embodiment, the receiver may be implemented separately from the speaker or as part thereof.

[0033] A display (160) (e.g., a display) can visually provide information to an external (e.g., a user) outside of the electronic device (101). The display (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling said device. According to one embodiment, the display (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of the force generated by said touch.

[0034] The audio module (170) can convert sound into an electrical signal or, conversely, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150) or output sound through the sound output module (155) or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphones) connected directly or wirelessly to the electronic device (101).

[0035] The sensor (176) can detect the operating state of the electronic device (101) (e.g., power or temperature) or the external environmental state (e.g., user state) and generate an electrical signal or data value corresponding to the detected state. According to one embodiment, the sensor (176) may include, for example, a gesture sensor, a gyroscope sensor, a barometric pressure sensor, a magnetic sensor, an accelerometer sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biosensor, a temperature sensor, a humidity sensor, or an illuminance sensor. For example, the sensor (176) may include an inertial measurement unit (IMU).

[0036] The interface (177) may support one or more specified protocols that can be used for the electronic device (101) to be connected directly or wirelessly to an external electronic device (e.g., electronic device (102)). According to one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0037] The connection terminal (178) may include a connector through which the electronic device (101) can be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0038] The haptic module (179) can convert an electrical signal into a mechanical stimulus (e.g., vibration or movement) or an electrical stimulus that can be perceived by the user through tactile or kinesthetic senses. According to one embodiment, the haptic module (179) may include, for example, a motor, a piezoelectric element, or an electric stimulation device.

[0039] The camera (180) can capture still images and video. According to one embodiment, the camera (180) may include one or more lenses, one or more image sensors, one or more image signal processors, or one or more flashes.

[0040] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented, for example, as at least part of a power management integrated circuit (PMIC).

[0041] The battery (189) can supply power to at least one component of the electronic device (101). According to one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0042] A communication module (190) can support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between an electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may include one or more communication circuits. The communication module (190) may include one or more communication processors (CP) that operate independently of a processor (120) (e.g., application processor) and support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., cellular communication module, short-range wireless communication module, or GNSS (global navigation satellite system) communication module) or a wired communication module (194) (e.g., LAN (local region network) communication module, or power line communication module). The corresponding communication module among these communication modules can communicate with an external electronic device (104) through a first network (198) (e.g., a short-range communication network such as Bluetooth, WiFi (wireless fidelity) direct or IrDA (infrared data relation)) or a second network (199) (e.g., a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., LAN or WAN)). These various types of communication modules may be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips).The wireless communication module (192) can identify or authenticate the electronic device (101) within a communication network, such as a first network (198) or a second network (199), using subscriber information (e.g., International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module (196).

[0043] The wireless communication module (192) can support 5G networks and next-generation communication technologies following 4G networks, for example, new radio access technology. NR access technology can support high-speed transmission of high-capacity data (enhanced mobile broadband (eMBB)), minimization of terminal power and connection of multiple terminals (massive machine type communications (mMTC)), or high reliability and low latency (ultra-reliable and low-latency communications (URLLC)). The wireless communication module (192) can support a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate, for example. The wireless communication module (192) can support various technologies for securing performance in the high-frequency band, such as beamforming, massive MIMO (multiple-input and multiple-output), full-dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large-scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), external electronic device (e.g., electronic device (104)), or network system (e.g., second network (199)). According to one embodiment, the wireless communication module (192) may support a Peak data rate (e.g., 20 Gbps or more) for eMBB realization, loss coverage (e.g., 164 dB or less) for mMTC realization, or U-plane latency (e.g., downlink (DL) and uplink (UL) each 0.5 ms or less, or round trip 1 ms or less) for URLLC realization.

[0044] An antenna module (197) can transmit a signal or power to or from an external source (e.g., an external electronic device). According to one embodiment, the antenna module (197) may include an antenna comprising a radiator made of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). According to one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as a first network (198) or a second network (199), may be selected from the plurality of antennas, for example, by a communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device through the selected at least one antenna. According to some embodiments, in addition to the radiator, other components (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as part of the antenna module (197).

[0045] According to various embodiments, the antenna module (197) may form a mmWave antenna module. According to one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent to a first surface (e.g., bottom surface) of the printed circuit board and capable of supporting a specified high frequency band (e.g., mmWave band), and a plurality of antennas (e.g., array antennas) disposed on or adjacent to a second surface (e.g., top surface or side surface) of the printed circuit board and capable of transmitting or receiving a signal of the specified high frequency band.

[0046] At least some of the above components can be connected to each other via a communication method between peripheral devices (e.g., bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)) and exchange signals (e.g., commands or data) with each other.

[0047] According to one embodiment, commands or data may be transmitted or received between an electronic device (101) and an external electronic device (104) through a server (108) connected to a second network (199).

[0048] Each of the external electronic devices (102, 104) and the server (108) may be the same or different type of device as the electronic device (101). According to one embodiment, all or part of the operations performed on the electronic device (101) may be performed on one or more of the external electronic devices (102, 104) or the server (108). For example, if the electronic device (101) needs to perform a function or service automatically or in response to a request from a user or another device, the electronic device (101) may request one or more external electronic devices to perform at least part of the function or service instead of performing the function or service itself or additionally. One or more external electronic devices that receive the request may perform at least part of the requested function or service, or additional functions or services related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may provide the result as is or additionally processed as at least part of the response to the request.

[0049] FIG. 2 is a diagram illustrating an example of an operation in which an electronic device according to various embodiments places a graphic object for displaying an image on a home screen.

[0050] An electronic device according to one embodiment (e.g., the electronic device (101) of FIG. 1) can display a home screen.

[0051] A home screen may refer to a screen containing graphic objects (e.g., application icons, widgets). Graphic objects placed on the home screen may be added and / or deleted (e.g., edited) by the user. According to one embodiment, the home screen may include a plurality of pages. A page may refer to an area displayed on the display of an electronic device at one time (e.g., at a certain timepoint) as at least part of the home screen.

[0052] According to one embodiment, each page of the home screen may be composed of a grid array having a plurality of rows and a plurality of columns. Each cell of the grid array may be used as a unit for arranging graphic objects. However, the home screen pages according to various embodiments of the present disclosure are not limited to being composed of a grid array, and at least one page of the home screen may be implemented as a gridless page. An example of a home screen including a gridless page is described in more detail below in FIG. 12.

[0053] In various embodiments of the present disclosure, the grid arrangement is a number of columns It can be expressed in the form of the number of rows. The home screen may have a grid array selected by the user from among multiple candidate grid arrays. For example, the multiple candidate grid arrays are 4 5 array, 4 6 array, 5 5 arrays, and 5 It may include 6 arrays. The electronic device is 5 by the user 6 Based on the selection of the array, the graphic object(s) included in a specific page 5 6. It can be arranged according to the grid array.

[0054] The application icons (e.g., icon (222), icon (223), icon (224), icon (225), icon (226), icon (227), icon (228), icon (229)) may include graphic objects for obtaining user input that directs the execution of the application (e.g., foreground execution). For example, the user may select an application that is frequently used from among the application(s) installed on the electronic device. The user may add the icon of the selected application to the home screen. The user may trigger the execution of the application (e.g., foreground execution) through the application icon displayed on the home screen.

[0055] A widget is a graphic object provided by a service corresponding to an application, and may include, for example, a graphic object for displaying information provided by the service. In various embodiments of the present disclosure, a widget may include a photo widget (e.g., a gallery widget) provided by a photo application (e.g., a gallery application).

[0056] A photo widget may refer to a widget for displaying image(s). According to one embodiment, a plurality of images may be displayed through a single photo widget. An electronic device may display one of the plurality of images (also referred to as a 'target image' in various embodiments of the present disclosure) in an area of ​​the home screen (or display) corresponding to the photo widget at a certain timepoint.

[0057] As will be described in more detail later, the electronic device can determine the display order of a plurality of images. The electronic device can display at least a portion of the image whose display order has been reached among the plurality of images in an area corresponding to a photo widget. For example, the electronic device can determine the image of the first display order as the target image. The electronic device can terminate the display of the target image through the photo widget based on the elapsed time (e.g., 2 seconds, 5 seconds) after starting the display of the target image. Based on the termination of the display of the target image, the electronic device can update the image of the next display order of the target image to the target image whose display order has been reached. The electronic device can start the display of the updated target image. The electronic device can repeat the updating of the target image and the display of the updated target image until the image of the last display order among the plurality of images is displayed as the target image. The electronic device can display a plurality of images sequentially according to the display order of the plurality of images. For example, based on the termination of the display of the last order image, the electronic device can select the image of the first display order among the plurality of images as the target image.

[0058] Referring to FIG. 2, on a screen (210), the electronic device may receive user input for selecting a widget (e.g., photo widget, gallery widget). The screen (210) may include a search bar (211) that provides a search function for supported widgets. For example, when the electronic device obtains a search term through the search bar (211), it may provide a search result that includes a widget associated with the search term among the supported widgets. A widget associated with a search term may include text associated with the widget (e.g., widget name, application name of the application providing the widget) and the search term matching at least partially.

[0059] The electronic device may display buttons (213, 214, 215, 216) that individually correspond to candidate sizes available for a widget (e.g., photo widget, gallery widget). The electronic device, candidate sizes (e.g., 1 1, 2 1, 2 2, and 4 2) Target size (e.g., 2 Based on detecting user input selecting a button (215) corresponding to 2), a widget (221) of the target size can be added.

[0060] As will be described in more detail later in FIG. 3, the widget (221) may have a frame of one of a plurality of candidate frame shapes. The plurality of frame shapes may include a regular frame shape (e.g., a rectangle, a rounded rectangle) and / or an irregular frame shape. For example, as shown in FIG. 2, 2 A widget having a size of 2 may have a frame with an irregular frame shape. Examples of regular frame shapes and irregular frame shapes are described in more detail later in FIG. 6.

[0061] According to one embodiment, the size of a graphic object (e.g., a widget (221)) may include the width and height of the area occupied by the graphic object in a grid array. The width may refer to the number of cells arranged along a first axis (e.g., the width). The height may be expressed as the number of cells arranged along a second axis (e.g., an axis perpendicular to the first axis). For example, the size of the graphic object is N In the case of M (where N and M are natural numbers greater than or equal to 2), a graphic object may be placed in an area where N columns and M rows intersect in a grid array.

[0062] In the screen (220), the electronic device can place (e.g., create, add) a widget (221) in an initial state (e.g., incomplete creation state) on the home screen. Referring to the screen (220) of FIG. 2, icons of applications (222, 223, 224, 225, 226, 227, 228, 229) can be placed on the home screen along with the widget (221). The state of the widget (221) being in an initial state (e.g., incomplete creation state) may mean that an image to be displayed through the widget (221) has not been selected. The electronic device may provide a function to select an image to be displayed through the widget (221) based on user input selecting the widget (221) in an initial state. User input for selecting a widget (221) may include, for example, a long press input, a single tap input (e.g., a click input), and / or a double tap input for the widget (221).

[0063] In the screen (230), the electronic device may display a screen for selecting an image to be displayed through the widget (221) based on user input selecting the initial state widget (221). For example, the screen for selecting an image may include button(s) for determining a unit for selecting an image (hereinafter also referred to as a 'selection unit'). For example, the electronic device may display a button (231) for selecting all image groups, a button (232) for selecting image(s) by image group (e.g., story), a button (233) for selecting image(s) by album, and a button (234) for selecting image(s) by image.

[0064] In various embodiments of the present disclosure, an 'image group' may mean the result of grouping images accessible to an electronic device based on semantic information of the images (e.g., objects appearing in the images) and / or metadata of the images (e.g., time of capture of the images, location of acquisition of the images). Images accessible to an electronic device may include images stored in the memory of the electronic device (e.g., memory (130) of FIG. 1) and / or images stored in a database accessible to the electronic device (e.g., server, cloud). For example, an image group may mean the result of grouping images accessible to the electronic device independently of user input (or based on an artificial intelligence model and / or metadata of the images). Grouping images stored in the electronic device and / or images accessible to the electronic device into one or more image groups may be performed by the electronic device and / or an external device (e.g., server (108) of FIG. 1).

[0065] In various embodiments of the present disclosure, 'album' may refer to the result of classifying images accessible by an electronic device based on the storage location of each image (e.g., a folder in which each image is stored). For example, the electronic device may group images into one or more albums by folder. The electronic device may determine that the image(s) contained in one folder constitute one album.

[0066] Although not explicitly illustrated in FIG. 2, the electronic device may select images included in each image group as images to be displayed via the widget (221) based on detecting user input selecting a button (231) for selecting all image groups. Based on user input selecting the button (231), the electronic device may skip the display of the screen (240) and display one of the selected images in the area corresponding to the widget (221).

[0067] In the screen (240), the electronic device may display a screen for selecting an image. Based on detecting user input that selects a button (e.g., button (232), button (233), button (234)) corresponding to the selection unit of the image, the electronic device may display a screen for selecting an image.

[0068] For example, if the electronic device determines an image group as a selection criterion based on user input selecting a button (232), it may display a screen where images can be selected on an image group basis (e.g., by image group).

[0069] For example, if the electronic device determines an album as the selection criterion for an image based on user input selecting a button (233), it may display a screen where images can be selected on an album basis.

[0070] For example, if the electronic device determines to select an image on an image-by-image basis based on user input selecting a button (234), it may display a screen where an image can be selected on an image-by-image basis (e.g., by image).

[0071] The electronic device can determine one or more images to be displayed through the widget (221) based on user input obtained through a screen (e.g., screen (240)) for selecting an image.

[0072] On the screen (250), the electronic device may display at least a portion of one of the selected images in an area corresponding to the widget (221) based on the completion of the selection of the images(s). In various embodiments of the present disclosure, when the selection of the images(s) to be displayed through the widget (221) is completed, the widget (221) may be represented as being in a completed state.

[0073] According to one embodiment, the electronic device may display selected image(s) in an area corresponding to the widget (221). If the image to be displayed through the widget (221) is selected as all image groups, the electronic device may display image(s) included in at least one image group in an area corresponding to the widget (221). If the image to be displayed through the widget (221) is selected as an image unit, the electronic device may display selected image(s) in an area corresponding to the widget (221). If the image to be displayed through the widget (221) is selected as an image group unit or an album unit, the electronic device may display images included in the selected image group or album in an area corresponding to the widget (221).

[0074] According to one embodiment, when a plurality of images are selected as images to be displayed through a widget (221), the electronic device may sequentially display a plurality of images in an area corresponding to the widget (221). For example, the electronic device may determine the display order of a plurality of images and display an image among the plurality of images for which the display order has been reached through the widget (221). Displaying a plurality of images through the widget (221) is described in more detail later in FIGS. 3 to 7.

[0075] FIG. 3 is a diagram illustrating an example of an operation in which an electronic device according to various embodiments changes the frame of a graphic object for displaying an image.

[0076] An electronic device according to one embodiment (e.g., the electronic device (101) of FIG. 1) can display at least a portion of an image through a graphic object (e.g., a widget (312)). The user can change the shape (e.g., frame shape) of the portion of the image displayed through the widget (312) (e.g., a photo widget, a gallery widget, the widget (221) of FIG. 2).

[0077] Referring to FIG. 3, on the screen (310), the electronic device can display an image (e.g., an image whose display order has been reached) through a frame of a first frame shape in an area corresponding to the widget (312). The electronic device can display a partial image of the image inside the frame in an area corresponding to the widget (312).

[0078] According to one embodiment, a frame may refer to an object for determining (e.g., cropping, extracting) a portion of an image to be displayed via a widget (312) among the images. The frame may have a specific shape. In various embodiments of the present disclosure, the shape of the frame may also be expressed as a frame shape. As illustrated in FIG. 3, the frame shape may include a regular frame shape (e.g., a rectangle, a rounded rectangle) and / or an irregular frame shape. Examples of regular frame shapes and irregular frame shapes are described in more detail later in FIG. 6.

[0079] On the screen (320), the electronic device may display a screen for editing the widget (312) based on user input (e.g., a long press on the widget (312)) that directs the editing of the widget (312). The screen for editing the widget (312) may include a button (321) for creating a widget stack, a button (322) for deleting the widget (312), and a button (323) for changing the settings of the widget (312).

[0080] According to one embodiment, creating a widget stack may include combining (e.g., nesting) another graphic object (e.g., another widget (not shown)) with a graphic object (e.g., widget (312)). For example, an electronic device may display a screen for adding another widget to be displayed as a stack nested in an area corresponding to a graphic object (e.g., widget (312)) based on user input selecting a button (321) for creating a widget stack. The electronic device may display one of a plurality of widgets combined (e.g., included) in the widget stack area. The electronic device may switch the widget to be displayed among the plurality of widgets combined (e.g., included) in the widget stack in response to user input (e.g., swipe input) obtained in the area corresponding to the widget stack. For example, the widget stack may include a first widget and a second widget. The electronic device may display the first widget in the area corresponding to the widget stack. The electronic device can switch the widget to be displayed in the area corresponding to the widget stack from the first widget to the second widget in response to a user's swipe input detected in the area corresponding to the widget stack.

[0081] In the screen (330), the electronic device may display a screen for changing the settings of the widget (312) based on user input selecting a button for changing the settings of the widget (312). The settings of the widget (312) may include an image to be displayed through the widget (312), a method for placing a frame on the image, and / or a setting for the frame shape of the frame to determine a partial image to be displayed through the widget (312). The method for placing a frame on the image may be determined (e.g., set) as one of center alignment (e.g., center crop), fit to widget size, and stretch to fill, for example. The electronic device may display buttons that individually correspond to a plurality of frame shapes. For example, the electronic device may display a button (301) corresponding to a first frame shape and a button (302) corresponding to a second frame shape.

[0082] On the screen (340), the electronic device can change the frame for displaying the image from a frame of the first frame shape to a frame of the second frame shape based on user input selecting a button (302) corresponding to the second frame shape. Based on determining the frame of the second frame shape, the electronic device can determine a portion of the image to be displayed through a widget (312) according to the second frame shape.

[0083] In the screen (350), the electronic device can display at least a portion of an image through a second frame-shaped frame in an area corresponding to the widget (312).

[0084] FIG. 4 is a drawing illustrating an example of a screen for changing the settings of a graphic object for displaying an image in an electronic device according to various embodiments.

[0085] According to one embodiment, an electronic device (e.g., the electronic device (101) of FIG. 1) may display a screen for changing the settings of a graphic object (e.g., a widget, a photo widget, a gallery widget) for displaying an image.

[0086] In the screen (410), the electronic device may display a screen for changing the settings of a widget, including a preview area (411), a button (412) for selecting an image to display via a widget (e.g., photo widget, gallery widget), a button (413) for automatic frame selection (e.g., auto frame), an area (414) for selecting a frame shape, a button (415) for adding a border object, a button (416) for adding a shadow object, and a button (417) for adjusting brightness.

[0087] As described above in FIG. 3, the area (414) may include a plurality of buttons that correspond individually to a plurality of frame shapes. Automatic frame selection may mean a function of selecting a frame shape corresponding to an image among a plurality of frame shapes. The operation of the electronic device determining a frame shape corresponding to an image is described in more detail later in FIG. 5 and FIG. 6.

[0088] In the screen (420), the electronic device may determine one or more frame shapes from a plurality of frame shapes as frame shapes for the widget based on user input selecting one or more buttons of the area (414). For example, in the screen (420), the electronic device may select six frame shapes as frame shapes for the widget. The electronic device may add a boundary object that surrounds the frame to the widget based on user input selecting a button (415).

[0089] The electronic device can display a first preview (422) for displaying an image through a frame of a first frame shape (401) among selected frame shapes in a preview area (421).

[0090] On the screen (430), the electronic device may display a second preview (432) through a frame of the second frame shape among the selected frame shapes, based on the selection of a plurality of frame shapes (e.g., first frame shape (401), second frame shape (402), third frame shape (403), fourth frame shape (404), fifth frame shape (405), and sixth frame shape (406)) as frame shapes for a widget in the preview area (431). For example, the electronic device may stop displaying the first preview (422) and start displaying the second preview (432) after a preset time (e.g., 2 seconds) has elapsed since the display of the first preview (422) began.

[0091] On the screen (440), the electronic device can select a frame shape corresponding to an image from among frame shapes (e.g., all frame shapes, one or more frame shapes selected based on user input) based on user input selecting a button (433) (or button (413)) for automatic frame selection. The frame shape corresponding to the image is described in more detail later in FIGS. 5 and 6.

[0092] FIG. 5 is a flowchart illustrating an example of a method for an electronic device according to various embodiments to generate a graphic object for displaying a plurality of images.

[0093] An electronic device according to one embodiment can add a graphic object for displaying a plurality of images to the home screen.

[0094] In operation (510), the electronic device may receive a request to create a graphic object for displaying multiple images. For example, the electronic device may receive user input selecting multiple images as images to be displayed through the graphic object (e.g., widget, photo widget, gallery widget).

[0095] As described above in FIGS. 2 to 4, the electronic device may select one or more images as images to be displayed through a single graphic object (e.g., widget). The electronic device may select a plurality of frame shapes and / or enable automatic frame selection. For example, the electronic device may perform an operation (e.g., operation (520)) of determining a frame shape corresponding to each image among the frame shapes (e.g., all frame shapes, one or more frame shapes selected based on user input) based on the activation of automatic frame selection.

[0096] In operation (520), the electronic device can determine, for each of the plurality of images, a frame shape corresponding to the image based on an object detected in the image. The electronic device can determine that each image corresponds to one frame shape. The electronic device can determine that two or more images correspond to one frame shape.

[0097] According to one embodiment, an electronic device can detect an object appearing in an image from an image (e.g., one of a plurality of images). The electronic device can select a frame shape corresponding to the image from among a plurality of candidate frame shapes based on the similarity between each candidate frame shape and the shape of the object region in the image where the object is detected. The candidate frame shape(s) are frame shape(s) selectable as frame shapes corresponding to the image, and can be determined as all frame shapes supported by the electronic device or as a frame shape selected based on user input among the frame shapes.

[0098] For each image, the electronic device can calculate the similarity between the object region of the image and each candidate frame shape. For example, the electronic device can determine the candidate frame shape having the greatest similarity among a plurality of candidate frame shapes as the frame shape corresponding to the image.

[0099] For example, an electronic device may calculate similarity between features based on the Intersection over Union (IoU) and / or Mean Squared Error (MSE) between features. The electronic device may determine a bounding box corresponding to each feature. The electronic device may adjust the features (e.g., scale at least one bounding box) so that the bounding boxes between two features have corresponding sizes. In various embodiments of the present disclosure, the size of the bounding box may be expressed as the number of pixels placed along a first axis (e.g., horizontal) and / or the number of pixels placed along a second axis (e.g., an axis perpendicular to the first axis) (e.g., vertical). The electronic device may determine similarity between the adjusted features. For example, the electronic device may determine similarity based on the Intersection over Union (IoU) and / or Mean Squared Error (MSE) between the adjusted features. The IoU and / or MSE between the adjusted features may be calculated with the center points of the bounding boxes of the adjusted features placed at the same location.

[0100] For example, the electronic device may adjust the size of the first shape and / or the second shape such that, when the aspect ratio (e.g., width-to-height ratio) of the first bounding box of the first shape is the same as the aspect ratio of the second bounding box of the second shape, the width and height of the first bounding box are each equal to the width and height of the second bounding box. The electronic device may determine the similarity between the first shape and / or the second shape adjusted to have the same size.

[0101] For example, if the aspect ratio (e.g., width-to-height ratio) of a first bounding box of a first shape is different from the aspect ratio of a second bounding box of a second shape, the electronic device may adjust the size of a first shape and / or a second shape such that one of the width and height (e.g., width) of the first bounding box is equal to the corresponding one of the width and height (e.g., width) of the second bounding box. The electronic device may determine a first partial similarity between the adjusted first shape and / or the adjusted second shape. The electronic device may adjust the size of the first shape and / or the second shape such that the other of the width and height (e.g., height) of the first bounding box is equal to the other of the width and height (e.g., height) of the second bounding box. The electronic device may determine a second partial similarity between the adjusted first shape and / or the adjusted second shape. The electronic device can determine the similarity between a first shape and a second shape based on a first part similarity and a second part similarity (e.g., as the average of the first part similarity and the second part similarity).

[0102] For example, an electronic device may calculate the similarity between shapes based on a shape similarity determination model. The shape similarity determination model may refer to a model created and / or trained to output output data corresponding to the similarity between the first shape and the second shape from input data corresponding to the first shape and the second shape. The shape similarity determination model may be implemented based on a machine learning model, a neural network (e.g., a convolutional neural network (CNN)), and / or a transformer. For example, the electronic device may determine the similarity between the object region and the candidate frame shape based on the result of applying the shape similarity determination model to the shape of the object region of an image and the shape of the candidate frame.

[0103] In FIG. 5, the electronic device primarily describes selecting a frame shape corresponding to an image based on the shape of an object region of the image, but is not limited thereto. According to various embodiments of the present disclosure, the electronic device may determine a frame shape corresponding to an image based on the category of an object detected in the image and / or the composition in which the object is placed in the image. An example of an operation to determine a frame shape based on the category of an object and / or the composition of an object is described in more detail later in FIG. 6.

[0104] In operation (530), the electronic device can determine the display order of multiple images based on multiple frame shapes corresponding to multiple images.

[0105] According to one embodiment, an electronic device can determine the order of a plurality of frame shapes. The electronic device can determine the display order of a plurality of images by sequentially arranging the image(s) corresponding to each of the plurality of frames. The determination of the order of the plurality of frame shapes and / or the determination of the display order of the plurality of images is described in more detail later in FIG. 7.

[0106] In operation (540), the electronic device can determine at least a portion of an image as a partial image based on an object region where an object is detected in each of the plurality of images and a frame shape corresponding to the image.

[0107] According to one embodiment, an electronic device may place a frame having a frame shape corresponding to the image on each image. For example, the electronic device may determine the size of the frame (e.g., adjust, scale) so that the frame having a frame shape covers (e.g., include, surround) an area corresponding to an object area in the image. The area corresponding to the object area may be an object area in the image where an object appears, or an area including an object area and a margin area surrounding the object area. The electronic device may place a frame on the image such that a frame having a determined size and frame shape covers the object area. In various embodiments of the present disclosure, placing a frame on the image may include determining an area corresponding to the frame in the image. Based on the placement of the frame on the image, the electronic device may determine an area inside the frame in the image as a partial image.

[0108] According to one embodiment, the electronic device can determine a partial image including a portion of an area outside the frame in an image by placing the frame on the image such that the frame covers a portion of an object area based on the satisfaction of a preset condition (e.g., a frame protrusion condition). An example of the operation of determining a partial image including an area placed outside the frame is described in more detail later in FIG. 8.

[0109] According to one embodiment, the electronic device may generate a supplementary image to supplement a portion of the frame based on the existence of a padding area within the frame that does not overlap with the image. An example of the operation of generating the supplementary image is described in more detail later in FIG. 9.

[0110] In various embodiments of the present disclosure, the electronic device is primarily described as determining a partial image based on an object region, but is not limited thereto. For example, the electronic device may determine a partial image based on whether the image is in focus or out of focus. Based on the image being in focus, the electronic device may determine a partial image based on a region corresponding to the focus. Based on the image being out of focus, the electronic device may determine a partial image based on a region surrounding a reference point of the image (e.g., a center point).

[0111] In operation (550), the electronic device can display a partial image of a target image, in which a determined display order among a plurality of images has been reached, through a frame of a frame shape corresponding to the target image in an area corresponding to a graphic object on the screen of the display.

[0112] As described above in FIG. 2, the electronic device may determine an image among a plurality of images whose display order has been reached as the target image. The electronic device may display at least a portion of the target image (e.g., a partial image of the target image) in an area corresponding to a graphic object through a frame having a frame shape corresponding to the target image. The electronic device may terminate the display of the target image based on the elapsed of a preset time (e.g., 2 seconds, 5 seconds) after starting the display of the target image. The electronic device may update the image among the plurality of images that has the next display order after the display order of the target image as the target image. The electronic device may repeat the operation to display at least a portion of the updated target image.

[0113] FIG. 6 is a diagram illustrating an example of an operation in which an electronic device according to various embodiments determines a frame shape corresponding to an image from a set of candidate frame shapes.

[0114] According to one embodiment, an electronic device (e.g., the electronic device (101) of FIG. 1) may select a frame shape corresponding to an image from among a plurality of candidate frame shapes. In various embodiments of the present disclosure, the plurality of candidate frame shapes may also be represented as a set of candidate frame shapes (600). The plurality of candidate frame shapes may include all of the frame shapes supported by the electronic device, as described above in FIG. 4 and FIG. 5, or one or more frame shapes selected based on user input from among the frame shapes.

[0115] Referring to FIG. 6, a set of candidate frame shapes (600) may include a first candidate frame shape (601), a second candidate frame shape (602), a third candidate frame shape (603), a fourth candidate frame shape (604), a fifth candidate frame shape (605), a sixth candidate frame shape (606), a seventh candidate frame shape (607), an eighth candidate frame shape (608), a ninth candidate frame shape (609), a tenth candidate frame shape (610), an eleventh candidate frame shape (611), and a twelfth candidate frame shape (612). The frame shapes (or candidate frame shapes) illustrated in FIG. 6 are merely examples, and the set of candidate frame shapes (600) may include other frame shapes.

[0116] The electronic device can obtain information regarding each candidate frame shape included in the candidate frame shape set (600), such as the composition mapped to the candidate frame shape, the category of the subject mapped to the candidate frame shape, the aspect ratio of the content mapped to the candidate frame shape, and / or the visual property of the image.

[0117] For example, identification information for each candidate frame shape can be tagged or labeled with information regarding the arrangement of objects suitable for the corresponding candidate frame shape. The arrangement of objects suitable for a specific candidate frame shape is selected by a user or a service provider, and the electronic device can map information regarding the arrangement selected by the user or the service provider to the specific candidate frame shape.

[0118] For example, identification information for each candidate frame shape can be tagged or labeled with information regarding the category of a subject (or object) suitable for the corresponding candidate frame shape. The category of an object suitable for a specific candidate frame shape is selected by a user or a service provider, and the electronic device can map information regarding the category of an object selected by the user or the service provider to the specific candidate frame shape.

[0119] For example, identification information for each candidate frame shape may be tagged or labeled with information regarding the aspect ratio of content suitable for the corresponding candidate frame shape. The aspect ratio of the content may include the aspect ratio of an image or the aspect ratio of an object area within an image. For example, the aspect ratio of content suitable for a specific candidate frame shape may be determined based on the aspect ratio of the candidate frame shape. For example, the aspect ratio of content suitable for a specific candidate frame shape is selected by a user or a service provider, and the electronic device may map information regarding the aspect ratio of the content selected by the user or the service provider to the specific candidate frame shape.

[0120] For example, identification information for each candidate frame shape may be tagged or labeled with information regarding the visual attributes of an image suitable for that candidate frame shape. The visual attributes of the image may include hue, saturation, brightness, contrast, and / or texture appearing in the image. For example, the attributes of an image suitable for a specific candidate frame shape are selected by a user or service provider, and the electronic device may map the information regarding the visual attributes of the image selected by the user or service provider to the specific candidate frame shape.

[0121] The electronic device can determine (e.g., select) a frame shape corresponding to an image based on information regarding a plurality of candidate frame shapes.

[0122] According to one embodiment, the electronic device can determine a frame shape corresponding to the image from a set of candidate frame shapes (600) based on the composition in which an object is placed in each image. For example, the electronic device can detect one or more objects appearing in the image from the image. The electronic device can determine a composition in which one or more objects are placed in the image. For example, the electronic device can determine a composition in which one or more objects are placed as one of a plurality of candidate compositions. The plurality of candidate compositions may include at least one of a horizontal composition, a vertical-horizontal composition, a vertical composition, a left-right asymmetrical composition, a left-right symmetrical composition, a circular composition, an S-shaped composition, an up-down asymmetrical composition, an up-down symmetrical composition, a diagonal composition, a triangular composition, or a double diagonal composition. The electronic device can select a frame shape corresponding to the image from among the plurality of candidate frame shapes based on whether each candidate frame shape is mapped to the determined composition.

[0123] According to one embodiment, the electronic device can determine a frame shape corresponding to the image from a set of candidate frame shapes (600) based on the category of an object detected in each image. For example, the electronic device can detect an object appearing in the image from the image. The electronic device can determine the category of the detected object. The electronic device can select a frame shape corresponding to the image from a plurality of candidate frame shapes based on whether each candidate frame shape is mapped to the determined category.

[0124] According to one embodiment, the electronic device may determine a frame shape corresponding to the image or the object region of the image from among a set of candidate frame shapes (600) based on the aspect ratio of each image or the object region of each image. For example, the electronic device may determine the aspect ratio of the object region of the image or the image. The electronic device may determine the candidate frame shape mapped to the aspect ratio of the object region of the image or the image as the frame shape corresponding to the image.

[0125] According to one embodiment, the electronic device may determine a frame shape corresponding to the visual attribute of the image from among a set of candidate frame shapes (600) based on the visual attribute of each image. For example, the electronic device may determine the visual attribute of the image (e.g., average saturation). Based on the fact that the average saturation of the image is greater than or equal to a threshold saturation (e.g., based on the fact that the image is composed of colors that are the same as or similar to primary colors), the electronic device may determine a candidate frame shape (e.g., a first candidate frame shape (601)) mapped to the average saturation being greater than or equal to the threshold saturation as the frame shape corresponding to the image.

[0126] According to various embodiments of the present disclosure, an electronic device may determine a frame shape corresponding to an image based on at least one of the shape of an object region of an image, the composition in which an object is placed in an image, the category of an object detected in an image, the aspect ratio of content (e.g., image, object region), or visual attributes of an image.

[0127] For example, the electronic device may select candidate frame shapes from a set of candidate frame shapes (600) that have a similarity greater than or equal to a threshold similarity with the shape of the object region. The electronic device may determine a frame shape corresponding to an image by comprehensively considering at least one of the composition in which the object is placed in the image, the category of the object detected in the image, the aspect ratio of the content (e.g., image, object region), or the visual attributes of the image, among the selected candidate frame shapes.

[0128] According to one embodiment, for each candidate frame shape, a composition score indicating the degree of suitability of the candidate frame shape according to composition, a category score indicating the degree of suitability of the candidate frame shape according to the category of an object, an aspect ratio score indicating the degree of suitability of the candidate frame shape according to the aspect ratio of the content, and / or an attribute score indicating the degree of suitability of the candidate frame shape according to image attributes may be predetermined and / or stored.

[0129] The electronic device can determine a score for a candidate frame shape based on a composition score, a category score, an aspect ratio score, and / or an attribute score for each of the candidate frame shapes selected based on the shape of the object region. The electronic device can determine a frame shape corresponding to an image based on the scores of the candidate frame shapes.

[0130] For example, for the first candidate frame shape (601), the composition score for the horizontal composition is 1 point, the composition score for the vertical composition is 1 point, and the composition score for both diagonal compositions is 5 points. For the first candidate frame shape (601), the score for the food category of the object is 4 points, and the score for the face category of the first candidate frame shape (601) is 1 point, and these scores may be stored. For the first candidate frame shape (601), the aspect ratio score for the content with an aspect ratio of 1:1 is 5 points, the aspect ratio score for the aspect ratio of 3:4 is 2 points, and the aspect ratio score for the aspect ratio of 4:3 is 2 points, and these scores may be stored. The electronic device can determine the score of the first candidate frame shape (601) for a specific image by summing the composition score based on the composition of an object detected in a specific image, the category score based on the category of an object detected in a specific image, and the aspect ratio score based on the aspect ratio of the specific image (or the object region of the specific image).

[0131] In various embodiments of the present disclosure, the electronic device primarily describes determining (e.g., selecting) a frame shape corresponding to an image from among primarily predetermined candidate frame shapes, but is not limited thereto.

[0132] According to one embodiment, the electronic device can generate a recommended frame shape based on an object detected in an image. The electronic device can add the generated recommended frame shape to a set of candidate frame shapes (600). For example, the electronic device can generate a recommended frame shape capable of covering an object area (or an area corresponding to the object area) in which an object is detected in the image. The electronic device can determine the similarity between each candidate frame shape of the set of candidate frame shapes (600) and the object area. If there are no candidate frames with a similarity greater than or equal to a threshold in the set of candidate frame shapes (600), the electronic device can generate a recommended frame shape based on the object area. The electronic device can determine the generated recommended frame shape as a frame shape corresponding to the image. The electronic device can generate information regarding the generated recommended frame shape (e.g., the category of the object suitable for the recommended frame shape, the composition of the object, the aspect ratio of the content, and / or the visual attributes of the image). The electronic device can obtain information regarding the generated recommended frame shape using a shape information generation model. The operation of the electronic device using the shape information generation model is described in more detail below. The electronic device can add a recommended frame shape to a set of candidate frame shapes (600). The electronic device can store information regarding the recommended frame shape along with identification information of the recommended frame shape.

[0133] According to one embodiment, the electronic device may generate additional frame shapes based on user input. For example, the electronic device may generate additional frame shapes having shapes indicated by user input. The electronic device may add the generated additional frame shapes to a set of candidate frame shapes (600). The electronic device may determine the category of an object or the composition of an object corresponding to the additional frame shapes from the generated additional frame shapes. For example, the electronic device may determine information about the additional frame shapes (e.g., the category of an object suitable for the additional frame shapes, the composition of an object, the aspect ratio of the content, and / or visual properties of the image) using a shape information generation model. The shape information generation model may refer to a model that is generated and / or trained to output output data corresponding to information about the frame shapes from input data corresponding to the frame shapes (e.g., additional frame shapes, recommended frame shapes). The shape information generation model may be implemented based on a machine learning model, a neural network (e.g., CNN), and / or a Transformer. The electronic device may store the additional frame shapes along with information about the determined category of an object or the composition of an object.

[0134] FIG. 7 is a diagram illustrating an example of an operation in which an electronic device according to various embodiments determines the display order of a plurality of images.

[0135] According to one embodiment, the electronic device can determine the display order (730) of a plurality of images based on a plurality of frame shapes corresponding to a plurality of images.

[0136] Referring to FIG. 7, the electronic device can determine, for each of the 10 images (e.g., first image (701), second image (702), third image (703), fourth image (704), fifth image (705), sixth image (706), seventh image (707), eighth image (708), ninth image (709), and tenth image (710)), a frame shape corresponding to the image among candidate frame shapes. For example, the electronic device can determine that the first frame shape (711) corresponds to the third image (703). The electronic device can determine that the second frame shape (712) corresponds to the first image (701). The electronic device can determine that the third frame shape (713) corresponds to the fifth image (705) and the ninth image (709). The electronic device can determine that the fourth frame shape (714) corresponds to the seventh image (707) and the eighth image (708). The electronic device can determine that the fifth frame shape (715) corresponds to the fourth image (704) and the tenth image (710). The electronic device can determine that the sixth frame shape (716) corresponds to the second image (702) and the sixth image (706).

[0137] According to one embodiment, the electronic device can determine the order (720) of a plurality of frame shapes corresponding to a plurality of images.

[0138] Referring to FIG. 7, the electronic device can obtain a sequence (720) of a plurality of candidate frame shapes.

[0139] According to one embodiment, the order (720) of a plurality of candidate frame shapes may be predetermined by an electronic device or an external device (e.g., a server (108) of FIG. 1). The electronic device may determine the order of a plurality of frame shapes (e.g., a first frame shape (711) to a sixth frame shape (716)) corresponding to a plurality of images according to the order (720) of a plurality of candidate frame shapes. For example, the order (720) of a plurality of frame shapes may be obtained in the order of a first frame shape (711), a second frame shape (712), a third frame shape (713), a fourth frame shape (714), a fifth frame shape (715), and a sixth frame shape (716).

[0140] According to one embodiment, the electronic device may determine the order (720) of a plurality of candidate frame shapes based on the user's usage history regarding the frame shapes. The user's usage history regarding the frame shapes may include the number of times or frequency with which each frame shape is selected by the user. For example, the order (720) of the plurality of candidate frame shapes may be determined in a descending order based on the number of times the candidate frame shapes are selected. For example, the order (720) of the plurality of candidate frame shapes may be determined in a descending order based on the frequency of selection of the candidate frame shapes.

[0141] According to one embodiment, the electronic device can determine the order (720) of a plurality of candidate frame shapes based on the number of image(s) corresponding to a plurality of candidate frame shapes. For example, the electronic device can determine the number of images corresponding to each candidate frame shape. The order (720) of the plurality of candidate frame shapes can be determined by sorting the candidate frame shapes in descending order according to the number of corresponding images.

[0142] According to one embodiment, the electronic device can determine the order (720) of a plurality of frame shapes based on similarity between a plurality of frame shapes. For example, the electronic device can determine a reference frame shape among a plurality of frame shapes. The electronic device can determine the order of the reference frame shape first. The electronic device can calculate the similarity between the reference frame shape and other frame shapes. The electronic device can determine the order of other frame shapes having the highest similarity to the reference frame shape as the next order after the reference frame shape. Based on determining the order of other frame shapes, the electronic device can update the reference frame shape to other frame shapes. Among the remaining frame shapes (e.g., frame shapes whose order has not been determined), the electronic device can determine the order of other frame shapes having the highest similarity to the updated reference frame shape as the next order after the reference frame shape. The electronic device can repeat updating the reference frame shape and determining the order of other frame shapes until the order of all the plurality of frame shapes is determined.

[0143] The electronic device can determine the display order (730) of a plurality of images so that one or more images corresponding to the same frame shape are arranged consecutively according to the order (720) of a plurality of determined frame shapes. For example, the electronic device can determine that the display order of two or more images corresponding to the same frame shape is consecutive.

[0144] For example, an electronic device may determine the first frame shape among a plurality of frame shapes as the target frame shape. The electronic device may determine the display order of the image corresponding to the target frame shape among a plurality of images as the first. If the target frame shape corresponds to two or more images, the electronic device may determine the display order of the two or more images sequentially starting from the first. For example, if the target frame shape corresponds to N image(s) (where N is a natural number greater than or equal to 1), the display order of the N image(s) may be determined as the first, ..., ..., Nth, respectively. The order of display among the N images corresponding to the target frame shape may be determined randomly and / or based on user input.

[0145] The electronic device may update the target frame shape to a frame shape having the next order of the target frame shape, based on determining the display order of the image(s) corresponding to the target frame shape. The electronic device may determine the display order of the image(s) corresponding to the target frame based on the update of the target frame shape. For example, when the target frame shape is updated, the display order of M images among a plurality of images may be determined. If the target frame shape corresponds to N image(s) (where N is a natural number greater than or equal to 1), the electronic device may determine the display order of the N image(s) as the M+1th, ... M+Nth, respectively. The electronic device may repeat the update of the target frame shape and the determination of the display order of the image corresponding to the target frame until the display order of all the plurality of images is determined.

[0146] Referring to FIG. 7, the electronic device can determine the display order of the third image (703) as the first based on the first frame shape (711). The electronic device can determine the display order of the first image (701) as the second based on the second frame shape (712). The electronic device can determine the display order of the fifth image (705) and the ninth image (709), respectively, as the third and fourth based on the third frame shape (713). The electronic device can determine the display order of the seventh image (707) and the eighth image (708), respectively, as the fifth and sixth based on the fourth frame shape (714). The electronic device can determine the display order of the fourth image (704) and the tenth image (710), respectively, as the seventh and eighth based on the fifth frame shape (715). The electronic device can determine the display order of the second image (702) and the sixth image (706), respectively, as the ninth and tenth based on the sixth frame shape (716).

[0147] In FIG. 7, the display order (730) of a plurality of images is primarily described as being determined based on the order (720) of a plurality of candidate frame shapes, but is not limited thereto. For example, an electronic device may display the display order (730) of a plurality of images based on the category of the images. For example, the category of the images may be determined according to an album or group of images containing the images. For example, a plurality of images included in the same album or the same group of images may be classified into the same category. For example, if a plurality of images are included in at least one of a first album and a second album, the electronic device may determine the display order (730) of a plurality of images to display the images included in the first album and then display the images included in the second album.

[0148] FIG. 8 is a diagram illustrating an example of an operation in which an electronic device according to various embodiments determines a partial image including at least a portion of an area outside the frame.

[0149] An electronic device according to one embodiment (e.g., the electronic device (101) of FIG. 1) can display at least a portion of an image (810) protruding into an area outside the frame based on the satisfaction of a frame protrusion condition.

[0150] The electronic device can place a frame of the frame shape (820) to cover a part of the object area (811) on the image (810) based on the fact that the object area (811) where an object is detected in the image (810) and the frame shape (820) corresponding to the image (810) satisfy the frame protrusion condition.

[0151] For example, the frame protrusion condition may include, in the initial placement of the frame on the image (810) determined based on the frame and object area (811) of the frame shape (820), the ratio of the area occupied by the object area (811) within the frame is less than or equal to a threshold ratio. The initial placement of the frame on the image (810) may mean the result of adjusting the size of the frame so that the frame of the frame shape (820) covers the area corresponding to the object area (811), and placing the frame of the adjusted size on the image (810).

[0152] For example, the frame protrusion condition may include the object detected in the image (810) being a single object (e.g., a cat) and / or including a protruding part of the object (e.g., an ear on the cat's face).

[0153] According to one embodiment, the electronic device can resize the frame and / or reposition the frame on the image (810) based on the object area (811) and the frame shape (820) satisfying the frame protrusion condition.

[0154] For example, the electronic device can determine a primary part (e.g., a person's face) and a secondary part (e.g., a hat worn by a person) within an object area (811). The electronic device can adjust the size of the frame so that the frame covers an area corresponding to the primary part (e.g., the primary part, the primary part, and a margin area surrounding the primary part). The electronic device can position the frame on the image (810) so that the frame covers an area corresponding to the primary part of the object within the image (810).

[0155] For example, the electronic device may determine a weight for each part (e.g., each pixel) of the object region (811). The electronic device may determine a weight for each part of the object region (811) using a weight determination model. The weight determination model may mean a model created and / or trained to output output data corresponding to the weight for each part of the object region (811) from input data corresponding to the object region (811). The weight determination model may be implemented based on a machine learning model, a neural network (e.g., CNN), and / or a Transformer. The electronic device may add a value based on the weight of a specific part to the objective function when a specific part of the object region (811) is located inside the frame. The electronic device may subtract a value based on the weight of a specific part from the objective function when a specific part of the object region (811) is located outside the frame. The electronic device may add a value based on the proportion of the object region (811) within the area inside the frame to the objective function. The electronic device can determine the size of the frame and / or the arrangement of the frame to maximize the value of the objective function.

[0156] The electronic device may determine the area (832) inside the frame of the image (810) and the protruding object area (831) outside the frame of the object area (811) as a partial image (830). Based on the display order of the image (810), the electronic device may display the partial image (830) in an area corresponding to a graphic object (e.g., a widget).

[0157] FIG. 9 is a drawing illustrating an example of an operation in which an electronic device according to various embodiments generates a supplementary image to supplement at least a portion of the internal area of ​​a frame.

[0158] An electronic device according to one embodiment (e.g., the electronic device (101) of FIG. 1) can generate a supplementary image (950) to supplement a padding area (940) based on the existence of a padding area within a frame. The padding area (940) may refer to an area within a frame that does not overlap with an image (910).

[0159] Referring to FIG. 9, for example, an electronic device may place a frame of the frame shape (920) on an image (910) based on an object region and a frame shape (920) in the image (910). The electronic device may determine an area inside the frame in the image (910) as a partial image (930).

[0160] As shown in FIG. 9, among the regions inside the frame, the first region (941) and the second region (942) may not overlap with the image (910). In other words, among the regions inside the frame, there may be a padding region (940) that includes the first region (941) and the second region (942).

[0161] The electronic device may generate a supplementary image (950) corresponding to a padding area (940) based on the fact that the area inside the frame includes a padding area (940) other than the determined partial image (910). For example, the electronic device may generate a first partial supplementary image (951) for a first area (941) and a second partial supplementary image (952) for a second area (942).

[0162] The electronic device can display a partial image (910) and a supplementary image (950) through a frame in an area corresponding to a graphic object (e.g., widget, photo widget, gallery widget) based on the display order of the image (910).

[0163] FIG. 10 is a drawing illustrating an example of an operation to change the frame shape when the aspect ratio of an area corresponding to a graphic object is changed in an electronic device according to various embodiments.

[0164] An electronic device according to one embodiment (e.g., the electronic device (101) of FIG. 1) can change the frame shape corresponding to an image based on a change in the aspect ratio of the area corresponding to a graphic object.

[0165] According to one embodiment, changing the aspect ratio of an area corresponding to a graphic object may include changing a display mode (e.g., landscape mode, portrait mode) indicating the display direction of the home screen according to the rotation of the electronic device. The aspect ratio of an area corresponding to a graphic object may refer to the ratio between the number of pixels arranged along a first axis (e.g., number of horizontal pixels) and the number of pixels arranged along a second axis (e.g., number of vertical pixels).

[0166] According to one embodiment, a graphic object may be placed on the home screen of a second display in response to a graphic object being placed on the home screen of a first display. In various embodiments of the present disclosure, when the home screen of the first display and the home screen of the second display are synchronized, the addition, deletion, and / or movement of a graphic object on the home screen of the first display may be reflected on the home screen of the second display. Due to differences between the first display (e.g., resolution, aspect ratio, grid arrangement) and the second display (e.g., resolution, aspect ratio, grid arrangement), the aspect ratio of the area corresponding to the graphic object added to the home screen of the first display and the aspect ratio of the area corresponding to the graphic object placed on the home screen of the second display may be different from each other.

[0167] For example, the first display may be a display included in a first electronic device, and the second display may be a display included in a second electronic device separate from the first electronic device. For example, the first display and the second display may be multiple displays included in a single electronic device. The first display may be a cover display of a foldable electronic device, and the second display may be a main display of a foldable electronic device. Changing the aspect ratio of a graphic object may include the display displaying the home screen switching between the first display and the second display.

[0168] Referring to FIG. 10, on a screen (1010), the electronic device may display a home screen while in a first display mode of the display (e.g., portrait mode). The electronic device may display at least a portion of an image through a frame in an area (1011) corresponding to a graphic object on the home screen. In various embodiments of the present disclosure, the aspect ratio of the area (1011) corresponding to a graphic object on the screen (1010) may be expressed as a first aspect ratio.

[0169] In the screen (1020), the electronic device can change the display mode of the display from a first display mode (e.g., portrait mode) to a second display mode (e.g., landscape mode). The electronic device can change the aspect ratio of the area (1021) corresponding to the graphic object in the home screen. In various embodiments of the present disclosure, the aspect ratio of the area (1021) corresponding to the graphic object in the screen (1020) may be expressed as a second aspect ratio.

[0170] The electronic device can change multiple frame shapes corresponding to multiple images based on the changed aspect ratio by changing the aspect ratio of an area corresponding to a graphic object in the display.

[0171] For example, if the aspect ratio of the region corresponding to the graphic object is a first aspect ratio, the electronic device may select a frame shape corresponding to each image from a first set of candidate frame shapes corresponding to the first aspect ratio. If the aspect ratio of the region corresponding to the graphic object is a second aspect ratio, the electronic device may again select (e.g., change) a frame shape corresponding to each image from a second set of candidate frame shapes corresponding to the second aspect ratio. For example, the electronic device may again select a frame shape corresponding to the image from among a plurality of candidate frame shapes included in the second set of candidate frame shapes based on an object detected in the image (e.g., object region, object category, object composition). Each candidate frame shape included in the second set of candidate frame shapes may be obtained as a result of transforming (e.g., stretching) a corresponding candidate frame shape from the first set of candidate frame shapes based on the second aspect ratio.

[0172] Referring to FIG. 10, the electronic device can change the frame shape corresponding to the image to a frame shape independent of the frame shape of the region (1011) (e.g., not obtainable through stretching) as a result of reselecting the frame shape corresponding to the image. In a portion of the screen (1031), the electronic device can display a partial image based on the reselected frame shape.

[0173] For example, the electronic device may replace the frame shape corresponding to each image as a result of deforming (e.g., stretching) it based on a second aspect ratio. Since the electronic device may only change the frame shape as a result of deforming the frame shape corresponding to each image and the image may not be deformed, the partial image may be changed according to the change in the frame shape.

[0174] Referring to FIG. 10, the electronic device can change the frame shape corresponding to the image to the frame shape obtained as a result of deforming (e.g., stretching) the frame shape corresponding to the image. In a portion of the screen (1032), the electronic device can display a partial image based on the deformed frame shape.

[0175] The electronic device can change a partial image for each of the plurality of images based on the changed frame shapes. As described above in FIGS. 2 through 9, or similarly, the electronic device can redetermine (e.g., change) the partial image to be displayed in the area corresponding to the graphic object using each image and the changed frame shape corresponding to that image.

[0176] The electronic device can display a modified partial image in an area corresponding to a graphic object through a frame of a modified frame shape.

[0177] However, although FIG. 10 mainly describes changing the frame shape corresponding to the image based on changing the aspect ratio of the area corresponding to the graphic object, it is not limited thereto. For example, the electronic device may maintain the frame shape and / or partial image based on changing the aspect ratio of the area corresponding to the graphic object. Referring to FIG. 10, in a part of the screen (1033), the electronic device may display a partial image through a frame of the frame shape in the area corresponding to the object, with the same frame shape and partial image as in the area (1011).

[0178] FIG. 11 is a diagram illustrating an example of an operation in which a plurality of graphic objects are displayed on a home screen in an electronic device according to various embodiments, at least partially overlapping.

[0179] An electronic device according to one embodiment (e.g., the electronic device (101) of FIG. 1) can determine a graphic object to be displayed on the top layer in the overlapping area based on the arrangement of a plurality of graphic objects so as to overlap at least partially on the home screen. For example, when a plurality of graphic objects are arranged so as to overlap at least partially on the home screen, the electronic device can create layers that individually correspond to the at least partially overlapped graphic objects. The electronic device can determine the relative positional relationship between the created layers. In various embodiments of the present disclosure, since the layers and graphic objects may have a one-to-one correspondence, the relative positional relationship between the created layers can be interpreted as substantially the same as the relative positional relationship between the graphic objects. For example, when the electronic device creates a first layer and a second layer, it can determine the first layer as the upper layer of the second layer (e.g., determine the second layer as the lower layer of the first layer).

[0180] According to one embodiment, an electronic device can determine the relative positional relationship between graphic objects (or layers) based on the object types of a plurality of graphic objects. The object type of a graphic object can be determined as one of a plurality of candidate object types. For example, the plurality of candidate object types may include at least one of an information provision type, an icon type, a button type, or a photo type. For example, a photo widget may be classified as a graphic object of the photo type. The electronic device can determine the relative positional relationship between a plurality of graphic objects based on the priority of the object types of the plurality of graphic objects. The electronic device can determine the first graphic object as a higher-level object than the second graphic object based on the fact that the priority of the object type of the first graphic object is higher than the priority of the object type of the second graphic object.

[0181] The priorities of multiple candidate object types can be predetermined. For example, the priority of an icon type may be higher than that of a button type. The priority of a button type may be higher than that of an information type. The priority of an information type may be higher than that of a photo type.

[0182] According to one embodiment, an electronic device can determine the relative positional relationship between graphic objects based on a user's usage history regarding a plurality of graphic objects (e.g., number of times the user uses the graphic object, frequency of use by the user). For example, if the number of times the user uses a first graphic object is greater than the number of times the user uses a second graphic object, the electronic device can determine the first graphic object as the parent object of the second graphic object.

[0183] According to one embodiment, an electronic device can determine the relative positional relationship between a first graphic object and a second graphic object based on whether an object area is displayed in an overlapping area between a first graphic object and a second graphic object. For example, if the overlapping area of ​​the first graphic object includes an object area and the overlapping area of ​​the second graphic object does not include an object area, the electronic device can determine the first graphic object as the parent object of the second graphic object.

[0184] Referring to FIG. 11, the electronic device may arrange a first photo widget (1111) and a second photo widget (1112) on a home screen so that they overlap at least partially. The electronic device may determine that the object area of ​​the image displayed through the first photo widget (1111) is included in the overlapping area that overlaps with the area corresponding to the second photo widget (1112) among the areas corresponding to the first photo widget (1111). The electronic device may determine that the object area of ​​the image displayed through the second photo widget (1112) is not included in the overlapping area that overlaps with the area corresponding to the first photo widget (1111) among the areas corresponding to the second photo widget (1112). The electronic device may determine the first photo widget (1111) as the upper object of the second photo widget (1112). The electronic device may display the first photo widget (1111) on the upper layer of the layer where the second photo widget (1112) is placed in the overlapping area.

[0185] In FIG. 11, the electronic device primarily describes determining the relative positional relationship of a plurality of graphic objects and arranging one graphic object so that it is at least partially obscured by another graphic object according to the determined relative positional relationship, but is not limited thereto.

[0186] According to one embodiment, an electronic device may combine and display a portion of an image (e.g., an object area) displayed through the second graphic object with the first graphic object, based on the fact that the first graphic object and the second graphic object are arranged to overlap at least partially, and the type of the first graphic object is an information providing type (or button type) and the type of the second graphic object is a photo type. In various embodiments of the present disclosure, a graphic object created through the combination between the first graphic object and the second graphic object may also be processed as a single graphic object (e.g., a combined object). For example, the electronic device may have the second graphic object being a widget for displaying a single image (e.g., a photo widget, a gallery widget) and / or the size of the second graphic object being equal to or smaller than a predetermined size (e.g., 2 Based on 1), an object area within an image can be determined as a partial image. An electronic device may place a second graphic object in an area related to the first graphic object (e.g., a surrounding area of ​​the first graphic object). The electronic device may display a partial image in an area corresponding to the second graphic object. For example, the electronic device may place the second graphic object so as to cover a part of the first graphic object (e.g., a non-main part). The electronic device may attach the partial image displayed through the second graphic object to the first graphic object in the form of a sticker.

[0187] According to one embodiment, the electronic device may adjust the size of at least one of the first graphic object and the second graphic object based on whether an object area corresponding to a preset object (e.g., a face) is displayed in the overlapping area between the first graphic object and the second graphic object. For example, the electronic device may reduce the size of a subsequent graphic object placed on the home screen later than one of the first graphic object and the second graphic object. The electronic device may reduce the size of the subsequent graphic object by a ratio greater than a preset ratio (e.g., 70%) relative to the existing size. If an object area is displayed in the overlapping area even after the size of the subsequent graphic object has been adjusted, the electronic device may reduce the size of a preceding graphic object placed on the home screen earlier than the subsequent graphic object.

[0188] According to one embodiment, the electronic device may have a first graphic object of an information providing type and a second graphic object of a photo type already placed on the home screen, and may additionally place a third graphic object of a photo type on the home screen. The electronic device may place the third graphic object between the first graphic object and the second graphic object.

[0189] According to one embodiment, the electronic device can combine a first graphic object and a second graphic object based on the arrangement such that the first graphic object and the second graphic object overlap at least partially.

[0190] For example, combining a first graphic object and a second graphic object may mean merging at least a portion of the first graphic object and at least a portion of the second graphic object into a single graphic object.

[0191] For example, combining a first graphic object and a second graphic object may include nesting the first graphic object and the second graphic object into a single widget stack. As described above in FIG. 3, the widget stack may include a plurality of graphic objects. The electronic device may display one of the plurality of graphic objects in an area corresponding to the widget stack. In response to a user's swipe input corresponding to the widget stack, the electronic device may switch between the plurality of graphic objects to be displayed in the area corresponding to the widget stack.

[0192] According to one embodiment, the electronic device may alternately display the first graphic object and the second graphic object in an overlapping area between the first graphic object and the second graphic object, based on the arrangement such that the first graphic object and the second graphic object overlap at least partially. For example, the electronic device may display the first graphic object for a predetermined time in the overlapping area (e.g., the top layer of the overlapping area). After displaying the first graphic object in the overlapping area (e.g., the top layer of the overlapping area) for a predetermined time, the electronic device may display the second graphic object in the overlapping area (e.g., the top layer of the overlapping area). Similarly, the electronic device may display the first graphic object in the overlapping area (e.g., the top layer of the overlapping area) for a predetermined time, and then display the second graphic object in the overlapping area (e.g., the top layer of the overlapping area). However, in various embodiments of the present disclosure, the switching of graphic objects displayed in the overlapping area is not limited to being performed based on time (e.g., a timer), and the switching of graphic objects displayed in the overlapping area may also be performed in response to user input (e.g., user input selecting the overlapping area).

[0193] FIG. 12 is a drawing illustrating an example of an operation in which an electronic device according to various embodiments displays a home screen including a gridless page.

[0194] An electronic device according to one embodiment (e.g., the electronic device (101) of FIG. 1) may display a home screen including a gridless page. As described above in FIG. 2, the home screen (or a page of the home screen) may be implemented as a gridless page.

[0195] A gridless page may refer to a page where graphic objects placed on the home screen can be placed independently of a grid array (e.g., without a grid array). Graphic objects placed on a gridless page can occupy at least a portion of the page independently of a grid array (e.g., without a grid array).

[0196] Referring to FIG. 12, on the screen (1210), the electronic device can display a gridless page in which graphic objects are arranged independently of a grid array (e.g., without a grid array) as at least part of the home screen.

[0197] FIG. 13a is a drawing illustrating an example of an operation in which an electronic device according to various embodiments adds a shadow object to a graphic object.

[0198] An electronic device according to one embodiment (e.g., the electronic device (101) of FIG. 1) may add a shadow object (1312a) to a graphic object (e.g., a widget, a photo widget). The shadow object (1312a) may include an object for mimicking a shadow created by light emitted from a virtual light source reaching the frame of the widget. The shadow object (1312a) may be positioned to surround at least a portion of the outline of the frame.

[0199] An electronic device according to one embodiment may determine whether to add a shadow object (1312a) to a graphic object based on the visibility of a graphic object placed on a home screen. For example, the electronic device may determine the visibility score of a specific pixel based on the difference between the pixel value of a specific pixel placed on the outline of a frame and the pixel value of another pixel adjacent to the specific pixel and placed outside the frame. For example, the pixel value of a pixel may include a plurality of elements (e.g., R value, G value, B value). The difference between the pixel values ​​of a first pixel and a second pixel may be determined based on L1 norm, L2 norm, and / or cosine similarity. The electronic device may determine whether to add a shadow object (1312a) to a graphic object based on the visibility scores of a plurality of pixels placed on the outline of a frame. For example, the electronic device may decide to add a shadow object (1312a) to a graphic object based on the fact that representative values ​​of the visibility scores (e.g., average, median, minimum) are below a threshold.

[0200] Referring to FIG. 13a, on the screen (1310a), the electronic device can display a widget (1311a) without a shadow object (1312a). The electronic device can decide to add a shadow object (1312a) to the widget (1311a) based on the visibility of the widget (1311a) (e.g., visibility scores of frames placed at the outer edge of the frame).

[0201] On the screen (1320a), the electronic device may display the widget (1311a) together with the shadow object (1312a). The electronic device may place and / or display the shadow object (1312a) at a position aligned with at least a portion of the outer edge of the frame of the widget (1311a).

[0202] FIG. 13b is a diagram illustrating an example of an operation in which an electronic device according to various embodiments adds a boundary object to a graphic object.

[0203] An electronic device according to one embodiment (e.g., the electronic device (101) of FIG. 1) may add a boundary object (1312b) to a graphic object (e.g., a widget, a photo widget, a gallery widget). The boundary object (1312b) may include an object that surrounds the outline of at least a portion of the graphic object. The boundary object (1312b) may be positioned to surround the outline of a frame.

[0204] An electronic device according to one embodiment may determine whether to add a boundary object (1312b) to a graphic object based on the visibility of a graphic object placed on a home screen. As described above in FIG. 13a or similarly, the electronic device may determine a visibility score for each of a plurality of pixels placed on the outer edge of a frame and determine whether to add a boundary object (1312b) to a graphic object based on the determined visibility scores.

[0205] Referring to FIG. 13b, on the screen (1310b), the electronic device may display a widget (1311b) without a boundary object (1312b). The electronic device may decide to add a boundary object (1312b) to the widget (1311b) based on the visibility of the widget (1311b) (e.g., visibility scores of frames placed outside the frame).

[0206] On the screen (1320b), the electronic device may display a widget (1311b) together with a boundary object (1312b). The electronic device may place and / or display a boundary object (1312b) surrounding the frame at a position aligned with the outer edge of the frame of the widget (1311b).

[0207] FIG. 13c is a drawing illustrating an example of an operation in which an electronic device according to various embodiments changes the visual properties of a graphic object.

[0208] An electronic device according to one embodiment (e.g., the electronic device (101) of FIG. 1) can change the visual properties of a graphic object (e.g., a widget, a photo widget, a gallery widget). The visual properties of the graphic object may include the hue, saturation, brightness, contrast, texture, and / or transparency of the graphic object.

[0209] An electronic device according to one embodiment may determine whether to change the visual properties of a graphic object based on the visibility of the graphic object placed on a home screen. As described above in FIG. 13a and / or FIG. 13b, or similarly, the electronic device may determine a visibility score for each of a plurality of pixels placed on the outer edge of a frame and determine whether to change the visual properties of the graphic object based on the determined visibility scores.

[0210] Referring to FIG. 13c, on the screen (1310c), the electronic device may display without changing the visual properties of the widget (1311c). For example, the electronic device may display at least a portion (e.g., a partial image) of the image displayed through the widget (1311c) as pixel values ​​of the image. The electronic device may decide to change the image properties of the widget (1311c) based on the visibility of the widget (1311c) (e.g., visibility scores of frames placed outside the frame).

[0211] On the screen (1320c), the electronic device can change the image properties of the widget (1311c). For example, the electronic device can display a partial image displayed through the widget (1311c) with the pixel values ​​of the image adjusted. For example, the electronic device can adjust the pixel values ​​to cause a change in at least one of the brightness, saturation, color, and / or contrast of the image. The electronic device can display the partial image with adjusted pixel values ​​in an area corresponding to a graphic object.

[0212] FIG. 14 is a drawing illustrating examples of frame shapes according to changes in the size of graphic objects according to various embodiments.

[0213] According to one embodiment, the size of a graphic object (e.g., widget, photo widget, gallery widget) can be configured in various ways. As described above in FIG. 3, the width and height representing the size of the graphic object can be expressed as the number of cells in a grid array.

[0214] According to one embodiment, the electronic device may obtain user input for changing the size of a graphic object. The electronic device may change the size of the graphic object. For example, the size of the graphic object is N N (where N is a natural number greater than or equal to 2) can be used. The electronic device, the size of the graphic object, is N In N, N It can be changed to (N+M)(where N is a natural number greater than or equal to 2, and M is a natural number greater than or equal to 1).

[0215] The electronic device can determine whether to change the frame shape corresponding to the image according to a change in the size of the graphic object. For each frame shape, the electronic device can store information indicating whether the frame shape is changed or maintained in response to a change in the size of the graphic object.

[0216] Referring to FIG. 14, for example, the first frame shape (1411), the second frame shape (1421), and the third frame shape (1431) can be changed to the first extended frame shape (1412), the second extended frame shape (1422), and the third extended frame shape (1432) in response to a change in the size of the graphic object.

[0217] For example, the fourth frame shape (1441), the fifth frame shape (1451), and the sixth frame shape (1461) can be maintained as a frame shape (1442) substantially the same as the fourth frame shape (1441), a frame shape (1452) substantially the same as the fifth frame shape (1451), and a frame shape (1462) substantially the same as the sixth frame shape (1461), independently of changes in the size of the graphic object (e.g., even when changes in the size of the graphic object occur).

[0218] An electronic device (101) according to one embodiment comprises: a display (160); at least one processor (120) including a processing circuit; and a memory (130) including one or more storage media for storing instructions. When the instructions are executed by the at least one processor (120), the electronic device (101) may obtain a request to create a graphic object for displaying a plurality of images, for each of the plurality of images, determine a frame shape corresponding to the image based on an object detected in the image, determine a display order (730) of the plurality of images based on the frame shapes corresponding to the plurality of images, and for each of the plurality of images, determine at least a part of the image as a partial image based on an object area where an object is detected in the image and a frame shape corresponding to the image, and display the partial image of the target image where the determined display order of the plurality of images has been reached through a frame of a frame shape corresponding to the target image in an area of ​​the screen of the display (160) corresponding to the graphic object.

[0219] When the above instructions are executed by the at least one processor (120), the electronic device (101) may be configured to detect an object appearing in an image from the image, and to select a frame shape corresponding to the image from among a plurality of candidate frame shapes based on the similarity between each candidate frame shape and the shape of the object region in the image where the object is detected.

[0220] When the above instructions are executed by the at least one processor (120), the electronic device (101) may be configured to detect one or more objects appearing in the image, determine a composition in which the one or more objects are arranged in the image, and select a frame shape corresponding to the image from among a plurality of candidate frame shapes based on whether each candidate frame shape is mapped to the determined composition.

[0221] When the above instructions are executed by the at least one processor (120), the electronic device (101) may be configured to detect an object appearing in the image from the image, determine the category of the detected object, and select a frame shape corresponding to the image from among a plurality of candidate frame shapes based on whether each candidate frame shape is mapped to the determined category.

[0222] When the above instructions are executed by the at least one processor (120), the electronic device (101) may be configured to determine the order (720) of the plurality of frame shapes corresponding to the plurality of images, and to determine the display order (730) of the plurality of images so that one or more images corresponding to the same frame shape are arranged consecutively according to the determined order (720) of the plurality of frame shapes.

[0223] When the above instructions are executed by the at least one processor (120), the electronic device (101) may be configured to place a frame of the frame shape to cover a part of the object area on the image based on the object area where an object is detected in the image and the frame shape corresponding to the image satisfying a frame protrusion condition, and to determine the area inside the frame in the image and the protruding object area outside the frame in the object area as the partial image.

[0224] When the above instructions are executed by the at least one processor (120), the electronic device (101) may be configured to place a frame of the frame shape on the image based on the object area where an object is detected in the image and the frame shape, determine the area inside the frame in the image as the partial image, and generate a supplementary image corresponding to the padding area based on the fact that the area inside the frame includes a padding area other than the determined partial image.

[0225] When the above instructions are executed by the at least one processor (120), the electronic device (101) may change the plurality of frame shapes corresponding to the plurality of images based on the changed aspect ratio of the area corresponding to the graphic object in the display (160), change the partial image for each of the plurality of images based on the changed frame shapes, and display the changed partial image in the area corresponding to the graphic object through the frame of the changed frame shape.

[0226] When the above instructions are executed by the at least one processor (120), the electronic device (101) may generate a recommended frame shape capable of covering an object area in which an object is detected in an image, and determine the generated recommended frame shape as a frame shape corresponding to the image.

[0227] When the above instructions are executed by the at least one processor (120), the electronic device (101) may be configured to generate an additional frame shape having a shape indicated by user input, determine a category of an object or a composition of an object corresponding to the generated additional frame shape from the additional frame shape, and store the additional frame shape together with information regarding the determined category of an object or a composition of an object.

[0228] According to one embodiment, a method performed by an electronic device may include: an operation of obtaining a request for the creation of a graphic object for displaying a plurality of images; an operation (510) of determining a frame shape corresponding to each of the plurality of images based on an object detected in the image; an operation (520) of determining the display order of the plurality of images based on the frame shapes corresponding to the plurality of images; an operation (530) of determining at least a part of the image as a partial image based on an object area where an object is detected in the image and a frame shape corresponding to the image for each of the plurality of images; and an operation (540) of displaying the partial image of the target image where the determined display order of the plurality of images has been reached through a frame of the frame shape corresponding to the target image in an area corresponding to the graphic object on the screen of the display of the electronic device.

[0229] The operation of determining the above frame shape may include: an operation of detecting an object appearing in the image from the image; and an operation of selecting a frame shape corresponding to the image from among a plurality of candidate frame shapes based on the similarity between each candidate frame shape and the shape of the object region in the image where the object is detected.

[0230] The operation of determining the above frame shape may include: an operation of detecting one or more objects appearing in the image from the image; an operation of determining a composition in which the one or more objects are arranged in the image; and an operation of selecting a frame shape corresponding to the image from among a plurality of candidate frame shapes based on whether each candidate frame shape is mapped to the determined composition.

[0231] The operation of determining the frame shape may include: an operation of detecting an object appearing in the image from the image; an operation of determining the category of the detected object; and an operation of selecting a frame shape corresponding to the image from among a plurality of candidate frame shapes based on whether each candidate frame shape is mapped to the determined category.

[0232] The operation of determining the display order of the plurality of images may include: the operation of determining the order of the plurality of frame shapes corresponding to the plurality of images; and the operation of determining the display order of the plurality of images such that one or more images corresponding to the same frame shape are continuously aligned according to the order of the determined plurality of frame shapes.

[0233] For each of the plurality of images above, the operation of determining at least a portion of the image as a partial image may include: the operation of arranging a frame of the frame shape to cover a portion of the object area on the image based on the fact that the object area where an object is detected in the image and the frame shape corresponding to the image satisfy a frame protrusion condition; and the operation of determining the area inside the frame in the image and the protruding object area outside the frame in the object area as the partial image.

[0234] For each of the plurality of images above, the operation of determining at least a portion of the image as a partial image may include: the operation of placing a frame of the frame shape on the image based on an object area where an object is detected in the image and the frame shape; the operation of determining an area inside the frame in the image as the partial image; and the operation of generating a supplementary image corresponding to the padding area based on the fact that the area inside the frame includes a padding area other than the determined partial image.

[0235] The above method may further include: an operation of changing the plurality of frame shapes corresponding to the plurality of images based on the changed aspect ratio of an area corresponding to the graphic object among the displays; an operation of changing a partial image for each of the plurality of images based on the changed frame shapes; and an operation of displaying the changed partial image in the area corresponding to the graphic object through a frame of the changed frame shape.

[0236] The operation of determining the above frame shape may include: an operation of generating a recommended frame shape capable of covering an object area in which an object is detected in an image; and an operation of determining the generated recommended frame shape as a frame shape corresponding to the image.

[0237] The electronic device according to the various embodiments disclosed in this document may be of various forms. The electronic device may include, for example, a portable communication device (e.g., a smartphone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a consumer electronics device. The electronic device according to the embodiments of this document is not limited to the devices described above.

[0238] The various embodiments of this document and the terms used therein are not intended to limit the technical features described in this document to specific embodiments, and should be understood to include various modifications, equivalents, or substitutions of said embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of said items unless the relevant context clearly indicates otherwise. In this document, phrases such as "A or B," "at least one of A and B," "at least one of A or B," "A, B or C," "at least one of A, B and C," and "at least one of A, B, or C" may each include any one of the items listed together in the corresponding phrase, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used simply to distinguish said components from other said components and do not limit said components in any other aspect (e.g., importance or order). Where any (e.g., 1st) component is referred to as "coupled" or "connected" to another (e.g., 2nd) component, with or without the terms "functionally" or "communicationly," it means that said any component may be connected to said other component directly (e.g., via a wire), wirelessly, or through a third component.

[0239] The term “module” as used in the various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit, for example. A module may be a component formed integrally, or a minimum unit of said component or a part thereof that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0240] Various embodiments of the present document may be implemented as software (e.g., program (140)) comprising one or more instructions stored in a storage medium (e.g., internal memory (136) or external memory (138)) readable by a machine (e.g., electronic device (101)). For example, a processor (e.g., processor (120)) of the machine (e.g., electronic device (101)) may call at least one of the one or more instructions stored in the storage medium and execute it. This enables the machine to be operated to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code that can be executed by an interpreter. The storage medium readable by the machine may be provided in the form of a non-transitory storage medium. Here, 'non-temporary' simply means that the storage medium is a tangible device and does not contain a signal (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily.

[0241] According to one embodiment, the method according to the various embodiments disclosed herein may be provided as included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or distributed online (e.g., download or upload) through an application store (e.g., Play Store™) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created on a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0242] According to various embodiments, each component (e.g., module or program) of the components described above may include a singular or multiple entities, and some of the multiple entities may be separated and placed in other components. According to various embodiments, one or more of the components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Generally or additionally, multiple components (e.g., module or program) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each of the multiple components in the same or similar manner as those performed by the corresponding component among the multiple components prior to integration. According to various embodiments, operations performed by the module, program, or other components may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

[0243] The embodiments described above may be implemented as hardware components, software components, and / or combinations of hardware and software components. For example, the devices, methods, and components described in the embodiments may be implemented using a general-purpose computer or a special-purpose computer, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing unit may execute an operating system (OS) and software applications executed on said operating system. Additionally, the processing unit may access, store, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, the processing unit may be described as being used as a single unit, but those skilled in the art will understand that the processing unit may include multiple processing elements and / or multiple types of processing elements. For example, the processing unit may include multiple processors or one processor and one controller. In addition, other processing configurations, such as parallel processors, are also possible.

[0244] Software may include computer programs, code, instructions, or a combination of one or more of these, and may configure a processing unit to operate as desired or command the processing unit independently or collectively. Software and / or data may be permanently or temporarily embodied in any type of machine, component, physical device, virtual equipment, or computer storage medium or device so as to be interpreted by the processing unit or to provide instructions or data to the processing unit. Software may be distributed over networked computer systems and may be stored or executed in a distributed manner. Software and data may be stored on computer-readable recording media.

[0245] The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc., either alone or in combination, and the program instructions recorded on the medium may be those specifically designed and configured for the embodiment or those known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory. Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc.

[0246] The hardware device described above may be configured to operate as one or more software modules to perform the operation of the embodiment, and vice versa.

Claims

1. In an electronic device (101), Display (160); At least one processor (120) including a processing circuit; and It includes a memory (130) comprising one or more storage media for storing instructions, and When the above instructions are executed by the at least one processor (120), the electronic device (101) is made to, Obtain a request to create a graphic object for displaying multiple images, and For each of the above plurality of images, a frame shape corresponding to the image is determined based on an object detected in the image, and Based on a plurality of frame shapes corresponding to the plurality of images, the display order (730) of the plurality of images is determined, and For each of the plurality of images above, at least a portion of the image is determined as a partial image based on an object region where an object is detected in the image and a frame shape corresponding to the image, and In the area corresponding to the graphic object on the screen of the display (160), a portion image of the target image at which the determined display order of the plurality of images has been reached is displayed through a frame of a frame shape corresponding to the target image. making, Electronic device (101).

2. In Paragraph 1, When the above instructions are executed by the at least one processor (120), the electronic device (101) is made to, From an image, detect an object appearing in the image, and Based on the similarity between each candidate frame shape and the shape of the object region in which the object is detected in the image, a frame shape corresponding to the image is selected from among a plurality of candidate frame shapes. making, Electronic device (101).

3. In any one of paragraphs 1 to 2, When the above instructions are executed by the at least one processor (120), the electronic device (101) is made to, From an image, detect one or more objects appearing in the image, and Determine the composition in which the above one or more objects are arranged in the above image, and Based on whether each candidate frame shape is mapped to the determined composition, a frame shape corresponding to the image is selected from among a plurality of candidate frame shapes. making, Electronic device (101).

4. In any one of paragraphs 1 through 3, When the above instructions are executed by the at least one processor (120), the electronic device (101) is made to, From an image, detect an object appearing in the image, and Determine the category of the detected object above, and Select a frame shape corresponding to the image from among a plurality of candidate frame shapes based on whether each candidate frame shape is mapped to the determined category. making, Electronic device (101).

5. In any one of paragraphs 1 through 4, When the above instructions are executed by the at least one processor (120), the electronic device (101) is made to, Determining the order (720) of the plurality of frame shapes corresponding to the plurality of images, and The display order (730) of the plurality of images is determined such that, according to the order (720) of the plurality of frame shapes determined above, one or more images corresponding to the same frame shape are arranged continuously. making, Electronic device (101).

6. In any one of paragraphs 1 through 5, When the above instructions are executed by the at least one processor (120), the electronic device (101) is made to, Based on the fact that an object region in which an object is detected in an image and a frame shape corresponding to the image satisfy a frame protrusion condition, a frame of the frame shape is placed on the image to cover a part of the object region, and The area inside the frame among the above images, and the protruding object area outside the frame among the object areas are determined as the partial image. making, Electronic device (101).

7. In any one of paragraphs 1 through 6, When the above instructions are executed by the at least one processor (120), the electronic device (101) is made to, Based on the object region where an object is detected in the above image and the frame shape, a frame of the frame shape is placed on the above image, and The area inside the frame among the above images is determined as the partial image, and Based on the fact that the area inside the frame includes a padding area other than the determined partial image, a supplementary image corresponding to the padding area is generated. making, Electronic device (101).

8. In any one of paragraphs 1 through 7, When the above instructions are executed by the at least one processor (120), the electronic device (101) is made to, Based on changing the aspect ratio of the area corresponding to the graphic object among the above displays (160), the plurality of frame shapes corresponding to the plurality of images are changed based on the changed aspect ratio, and Based on the above modified frame shapes, a partial image for each of the plurality of images is modified, and Display the modified partial image in the area corresponding to the graphic object through a frame of the modified frame shape. making, Electronic device (101).

9. In any one of paragraphs 1 through 8, When the above instructions are executed by the at least one processor (120), the electronic device (101) is made to, Generates a recommended frame shape capable of covering the object area where an object is detected in the image, and The above-generated recommended frame shape is determined as the frame shape corresponding to the image. making, Electronic device (101).

10. In any one of paragraphs 1 through 9, When the above instructions are executed by the at least one processor (120), the electronic device (101) is made to, Generate additional frame shapes having shapes indicated by user input, and From the additional frame shape generated above, determine the category of an object or the composition of an object corresponding to the additional frame shape, and Store the above additional frame shape together with information regarding the category of the determined object or the composition of the object. making, Electronic device (101).

11. In a method performed by an electronic device, An operation to obtain a request for the creation of a graphic object for displaying multiple images; For each of the above plurality of images, an operation (510) of determining a frame shape corresponding to the image based on an object detected in the image; An operation (520) for determining the display order of the plurality of images based on the plurality of frame shapes corresponding to the plurality of images; For each of the plurality of images above, an operation (530) of determining at least a part of the image as a partial image based on an object region where an object is detected in the image and a frame shape corresponding to the image; and Operation (540) of displaying a portion image of a target image, in which the determined display order of the plurality of images has been reached, through a frame of a frame shape corresponding to the target image in an area corresponding to the graphic object on the screen of the display of the electronic device. A method including 12. In Paragraph 11, The operation of determining the above frame shape is, From an image, an operation of detecting an object appearing in the image; and A method comprising selecting a frame shape corresponding to an image from among a plurality of candidate frame shapes based on the similarity between each candidate frame shape and the shape of the object region in which the object is detected in the image. method.

13. In any one of paragraphs 11 to 12, The operation of determining the above frame shape is, An operation to detect one or more objects appearing in an image from an image; An operation to determine a composition in which one or more of the objects are arranged in the above image; and A method comprising selecting a frame shape corresponding to the image from among a plurality of candidate frame shapes based on whether each candidate frame shape is mapped to the determined composition, method.

14. In any one of paragraphs 11 through 13, The operation of determining the above frame shape is, An operation to detect an object appearing in an image from an image; An operation to determine the category of the detected object; and A method comprising selecting a frame shape corresponding to the image from among a plurality of candidate frame shapes based on whether each candidate frame shape is mapped to the determined category, method.

15. A computer-readable recording medium storing one or more computer programs comprising instructions for performing the method of any one of paragraphs 11 through 14.

Images