Electronic device capable of changing state and driving method thereof

Abstract

An electronic device according to embodiments disclosed herein may comprise: a display module including a flexible display having a display size that can be expanded or reduced; a memory for storing instructions; and at least one processor operatively connected to the memory and the display module. The instructions, when executed individually and / or collectively by the at least one processor, may cause the electronic device to: determine a usage pattern of a user of the electronic device according to the usage environment using a first artificial intelligence model; detect the display size of the flexible display on the basis of a change in the state of the display module; and perform layout optimization on the basis of the usage pattern and the display size using a second artificial intelligence model.

Description

Electronic device capable of changing state and method of driving same

[0001] Various embodiments of the present disclosure relate to an electronic device capable of changing states, and more particularly, to an electronic device that performs layout optimization and a method of driving the same.

[0002] The variety of services and additional features offered through electronic devices (e.g., smartphones and other portable electronic devices) is steadily increasing. To meet the needs of various types of users and enhance their efficiency, telecommunications service providers and / or electronic device manufacturers are competitively developing electronic devices that offer a variety of features and differentiate themselves from competitors. Consequently, the functions offered by electronic devices are becoming increasingly sophisticated.

[0003] Recently, sliderable electronic devices with variable screen sizes, in addition to conventional bar-type devices, have been attracting attention. Sliderable electronic devices are designed to adjust the screen size (e.g., expand or contract) according to the user's needs, offering expanded usability compared to conventional bar-type electronic devices with fixed screen sizes. For example, users can perform simple tasks on a small screen (e.g., non-expanded) and multitask using various content on a larger screen.

[0004] In a sliderable electronic device, if the screen layout remains fixed when the screen size is expanded or reduced, the usability of the sliderable electronic device may be reduced. Therefore, adaptive and dynamic layout optimization according to changes in the screen size of the sliderable electronic device is required.

[0005] According to one embodiment, an electronic device and a method of driving the same can provide a user interface that is adaptive to various usage environments and user usage patterns by optimizing the layout of a home screen, an application (app) screen, or a lock screen as the display size expands or contracts.

[0006] According to one embodiment, an electronic device may include a display module including a flexible display having an expandable or contractible display size, a memory storing instructions, and at least one processor operatively connected to the memory and the display module. The instructions, when individually and / or collectively executed by the at least one processor, may cause the electronic device to perform at least one operation. The at least one operation may include an operation of determining a usage pattern of a user of the electronic device according to a usage environment using a first artificial intelligence model. The at least one operation may include an operation of detecting a display size of the flexible display based on a change in a state of the display module. The at least one operation may include an operation of performing layout optimization based on the usage pattern and the display size using a second artificial intelligence model.

[0007] According to embodiments of the present disclosure, a method for driving an electronic device may include at least one operation. The at least one operation may include an operation of determining a usage pattern of a user of the electronic device according to a usage environment using a first artificial intelligence model. The at least one operation may include an operation of detecting a display size of the flexible display based on a change in the state of the display module. The at least one operation may include an operation of performing layout optimization based on the usage pattern and the display size using a second artificial intelligence model.

[0008] According to one embodiment, the electronic device of the present disclosure and its driving method can efficiently use screen space by optimizing the layout of a home screen, an app screen, or a lock screen as the display size expands or contracts.

[0009] In addition, according to one embodiment, the electronic device and its driving method can provide a user interface that is adaptive to a situational usage scenario or a user's intention to change the screen size by analyzing various usage environments and learning a user's usage pattern using an artificial intelligence model.

[0010] Additionally, according to one embodiment, the electronic device and its driving method can notify a user of a change in a user interface according to layout optimization by displaying a layout optimization execution queue through a display when performing layout optimization.

[0011] Therefore, according to one embodiment, the electronic device and its driving method can increase the usability of the sliderable electronic device and maximize user convenience and user satisfaction.

[0012] The effects that can be obtained from the exemplary embodiments of the present disclosure are not limited to the effects mentioned above, and other effects not mentioned can be clearly derived and understood by those skilled in the art to which the exemplary embodiments of the present disclosure pertain from the following description. In other words, unintended effects resulting from implementing the exemplary embodiments of the present disclosure can also be derived by those skilled in the art from the exemplary embodiments of the present disclosure.

[0013] FIG. 1 is a block diagram illustrating an exemplary electronic device within a network environment according to one or more embodiments.

[0014] FIG. 2A is a top plan view of an exemplary electronic device in a first state.

[0015] FIG. 2b is a bottom view of an exemplary electronic device in a first state.

[0016] Figure 2c is a plan view of an exemplary electronic device within a second state.

[0017] FIG. 2d is a bottom view of an exemplary electronic device within a second state.

[0018] FIG. 3 is a flowchart illustrating exemplary operations of an electronic device according to one or more embodiments.

[0019] FIGS. 4A and 4B are exemplary diagrams illustrating layout optimization performed on a home screen of an electronic device according to one or more embodiments.

[0020] FIGS. 5A to 5C are exemplary diagrams illustrating layout optimization performed on an app screen of an electronic device according to one or more embodiments.

[0021] FIG. 6 is an exemplary diagram illustrating layout optimization performed on a lock screen of an electronic device according to one or more embodiments.

[0022] FIG. 7 is an exemplary diagram illustrating layout optimization for app icons in an electronic device according to one or more embodiments.

[0023] FIG. 8 is an exemplary diagram illustrating layout optimization for app foldering in an electronic device according to one or more embodiments.

[0024] FIG. 9 is an exemplary diagram illustrating layout optimization for a widget by an electronic device according to one or more embodiments.

[0025] FIGS. 10A and 10B are exemplary diagrams illustrating layout optimization for a wall paper of an electronic device according to one or more embodiments.

[0026] FIG. 11 is an exemplary diagram illustrating layout optimization considering a wallpaper image in an electronic device according to one or more embodiments.

[0027] FIG. 12 is an exemplary diagram illustrating layout optimization for a status bar of an electronic device according to one or more embodiments.

[0028] FIG. 13 is a flowchart illustrating an exemplary operation of an electronic device according to one or more embodiments to perform layout optimization based on a state change intent.

[0029] FIG. 14 is an exemplary diagram illustrating layout optimization performed by an electronic device while running a music app according to one or more embodiments.

[0030] FIG. 15 is an exemplary diagram illustrating layout optimization performed by an electronic device while running a calling app according to one or more embodiments.

[0031] FIG. 16 is an exemplary diagram illustrating layout optimization performed by an electronic device while executing an Internet app according to one or more embodiments.

[0032] FIGS. 17A and 17B are exemplary diagrams illustrating layout optimizations performed by an electronic device while running a navigation app according to one or more embodiments.

[0033] FIG. 18 is an exemplary diagram illustrating a settings window for layout optimization of an electronic device according to one or more embodiments.

[0034] FIG. 19 is a flowchart illustrating an operation of an electronic device according to one or more embodiments to display a layout optimization execution queue.

[0035] FIGS. 20A to 20D are exemplary diagrams illustrating a layout optimization execution queue of an electronic device according to one or more embodiments.

[0036] It should be understood that the various embodiments and terms used in this document are not intended to limit the technical features described in this document to specific embodiments, but rather to include various modifications, equivalents, or substitutes of the embodiments.

[0037] In connection with the description of the drawings, similar reference numerals may be used for similar or related components.

[0038] The singular form of a noun corresponding to an item may include one or more items, unless the context clearly indicates otherwise.

[0039] In this document, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" may include any one of the items listed together in that phrase, or all possible combinations thereof.

[0040] Terms such as "first," "second," or "first" or "second" may be used simply to distinguish one component from another and do not qualify the components in any other respect (e.g., importance or order).

[0041] When a component (e.g., a first component) is referred to as being “coupled” or “connected” to another component (e.g., a second component), with or without the terms “functionally” or “communicatively,” it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0042] The terms "include" and / or "have" are intended to specify the presence of a feature, number, step, operation, component, part or combination thereof described herein, but do not preclude the presence or addition of one or more other features, numbers, steps, operations, components, parts or combinations thereof.

[0043] When a component is said to be “connected,” “coupled,” “supported,” or “in contact with” another component, this includes not only cases where the components are directly connected, coupled, supported, or in contact, but also cases where the components are indirectly connected, coupled, supported, or in contact through a third component.

[0044] When we say that a component is "on" another component, this includes not only cases where the component is in contact with the other component, but also cases where there is another component between the two components.

[0045] The term "and / or" includes any combination of a plurality of related described elements or any one of a plurality of related described elements.

[0046] The operating principle and embodiments of the present invention will be described with reference to the attached drawings below.

[0047] FIG. 1 is a block diagram illustrating an exemplary electronic device (101) within a network environment (100) according to one or more embodiments.

[0048] Referring to FIG. 1, in a network environment (100), an electronic device (101) may communicate with an electronic device (102) via a first network (198) (e.g., a short-range wireless communication network), or may communicate with at least one of an electronic device (104) or a server (108) via a second network (199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (101) may communicate with the electronic device (104) via the server (108). According to one embodiment, the electronic device (101) may include a processor (120), a memory (130), an input module (150), an audio output module (155), a display module (160), an audio module (170), a sensor module (176), an interface (177), a connection terminal (178), a haptic module (179), a camera module (180), a power management module (188), a battery (189), a communication module (190), a subscriber identification module (196), or an antenna module (197). In some embodiments, the electronic device (101) may omit at least one of these components (e.g., the connection terminal (178)), or may have one or more other components added. In some embodiments, some of these components (e.g., the sensor module (176), the camera module (180), or the antenna module (197)) may be integrated into one component (e.g., the display module (160)).

[0049] The processor (120) may include various processing circuits and / or multiple processors. For example, the term "processor" as used in this disclosure, including the claims, may include various processing circuits, including at least one processor, one or more of which may be configured to individually and / or collectively perform the various functions described in this disclosure in a distributed manner. When the terms "processor," "at least one processor," and "one or more processors" as used in this disclosure are described as being configured to perform a number of functions, these terms encompass, for example, without limitation, situations where one processor performs some of the recited functions and other processor(s) perform other of the recited functions, as well as situations where a single processor may perform all of the recited functions. Furthermore, the at least one processor may include a combination of processors that perform various of the recited / disclosed functions, for example, in a distributed manner. The at least one processor may execute program instructions to achieve or perform the various functions. The processor (120) may, for example, execute software (e.g., a program (140)) to control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) and perform various data processing or operations. According to one embodiment, as at least a part of the data processing or operations, the processor (120) may store commands or data received from other components (e.g., a sensor module (176) or a communication module (190)) in a volatile memory (132), process the commands or data stored in the volatile memory (132), and store result data in a non-volatile memory (134).According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or an auxiliary processor (123) (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor) that can operate independently or together with the main processor (121). For example, when the electronic device (101) includes the main processor (121) and the auxiliary processor (123), the auxiliary processor (123) may be configured to use less power than the main processor (121) or to be specialized for a given function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as a part thereof.

[0050] The auxiliary processor (123) may control at least a portion of functions or states associated with at least one component (e.g., a display module (160), a sensor module (176), or a communication module (190)) of the electronic device (101), for example, on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (123) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (180) or a communication module (190)). In one embodiment, the auxiliary processor (123) (e.g., a neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. This learning can be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or can be performed through a separate server (e.g., server (108)). The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include multiple artificial neural network layers.The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to, or alternatively to, a hardware structure, an artificial intelligence model may include a software structure.

[0051] The memory (130) can store various data used by at least one component (e.g., processor (120) or sensor module (176)) of the electronic device (101). The data can include, for example, software (e.g., program (140)) and input data or output data for commands related thereto. The memory (130) can include volatile memory (132) or non-volatile memory (134).

[0052] The program (140) may be stored as software in the memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0053] The input module (150) can receive commands or data to be used in a component of the electronic device (101) (e.g., a processor (120)) from an external source (e.g., a user) of the electronic device (101). The input module (150) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0054] The audio output module (155) can output audio signals to the outside of the electronic device (101). The audio output module (155) can include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. In one embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

[0055] The display module (160) can visually provide information to an external party (e.g., a user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by the touch.

[0056] The audio module (170) can convert sound into an electrical signal, or vice versa, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150), output sound through the sound output module (155), or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphone) directly or wirelessly connected to the electronic device (101).

[0057] The sensor module (176) can detect the operating status (e.g., power or temperature) of the electronic device (101) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. According to one embodiment, the sensor module (176) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0058] The interface (177) may support one or more designated protocols that may be used to connect the electronic device (101) directly (e.g., wired) or wirelessly with an external electronic device (e.g., electronic device (102)). In one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0059] The connection terminal (178) may include a connector through which the electronic device (101) may be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0060] The haptic module (179) can convert electrical signals into mechanical stimuli (e.g., vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic sensations. According to one embodiment, the haptic module (179) can include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

[0061] The camera module (180) can capture still images and videos. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0062] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

[0063] A battery (189) may power at least one component of the electronic device (101). In one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0064] The communication module (190) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may operate independently from the processor (120) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (194) (e.g., a local area network (LAN) communication module, or a power line communication module). Among these communication modules, the corresponding communication module can communicate with an external electronic device (104) via a first network (198) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (199) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules can be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can verify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) by using subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (196).

[0065] The wireless communication module (192) can support 5G networks and next-generation communication technologies following the 4G network, such as NR access technology (new radio access technology). The NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and connection of multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency communications)). The wireless communication module (192) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The wireless communication module (192) can support various technologies for securing performance in a high-frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), an external electronic device (e.g., the electronic device (104)), or a network system (e.g., the second network (199)). According to one embodiment, the wireless communication module (192) can support a peak data rate (e.g., 20 Gbps or more) for eMBB realization, a loss coverage (e.g., 164 dB or less) for mMTC realization, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL), or 1 ms or less for round trip) for URLLC realization.

[0066] The antenna module (197) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). In one embodiment, the antenna module (197) may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). In one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (198) or the second network (199), may be selected from the plurality of antennas, for example, by the communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device via the at least one selected antenna. In some embodiments, in addition to the radiator, another component (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module (197).

[0067] According to various embodiments, the antenna module (197) may form a mmWave antenna module. In one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent a first side (e.g., a bottom side) of the printed circuit board and capable of supporting a designated high-frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) disposed on or adjacent a second side (e.g., a top side or a side side) of the printed circuit board and capable of transmitting or receiving signals in the designated high-frequency band.

[0068] At least some of the above components can be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)).

[0069] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) via a server (108) connected to a second network (199). Each of the external electronic devices (102 or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations executed in the electronic device (101) may be executed in one or more of the external electronic devices (102, 104, or 108). For example, when the electronic device (101) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (101) may, instead of or in addition to executing the function or service itself, request one or more external electronic devices to perform the function or at least a part of the service. One or more external electronic devices that receive the request may execute at least a portion of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may process the result as is or additionally and provide it as at least a portion of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used, for example. The electronic device (101) may provide an ultra-low latency service by using distributed computing or mobile edge computing, for example. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server utilizing machine learning and / or a neural network. According to one embodiment, the external electronic device (104) or the server (108) may be included in the second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0070] The display module (160) of the present disclosure may be flexible. The display module (160) may be a liquid-crystal display (LCD), a light-emitting diode (LED), an organic light-emitting diode (OLED), or any other suitable display known to those skilled in the art. The display module (160) may include a display area that provides at least a portion of the outer surface of the electronic device (101) and is visually exposed outside the housing of the electronic device (101). For example, since the display module (160) has flexibility, at least a portion of the display module (160) may be rollable into the housing or slidable into the housing. For example, the size of the display area may vary depending on the size of at least a portion of the display module (160) that is rolled into the housing or slid into the housing. For example, an electronic device (101) including a display module (160) may be in a plurality of states, including a first state providing the display area having a first size and a second state providing the display area having a second size different from the first size. For example, the first state may be exemplified through the description of FIGS. 2A and 2B.

[0071] FIG. 2A is a top plan view of an exemplary electronic device (101) in a first state.

[0072] In FIG. 2A, the electronic device (101) may include a first housing (210), a second housing (220) movable relative to the first housing (210) in a first direction (261) parallel to the y-axis or a second direction (262) parallel to the y-axis and opposite to the first direction (261), and a display (230) (e.g., the display module (160) of FIG. 1). The first housing (210) and the second housing (220) may be the same size or different sizes. In one or more examples, the first housing (210) and the second housing (220) may be purchased separately and connected to each other to form a flexible display. In one or more examples, the first housing (210) and the second housing (220) may have the same screen type. In one or more examples, the first housing (210) and the second housing (220) may have different screen types. For example, the first housing (210) may have an LED screen, and the second housing (220) may have an OLED screen. Although the second housing (220) is described as being moved relative to the first housing (210), it is not limited thereto. For example, the first housing (210) may be moved relative to the second housing (220). For example, depending on a change in the relative positional relationship between the first housing (210) and the second housing (220), the size of the display area of ​​the display (230) visually exposed outside the housing of the electronic device (101) may change.

[0073] For example, within the first state, the second housing (220) may be movable (e.g., change position) relative to the first housing (210) in a first direction (261) among the first direction (261) and the second direction (262). For example, within the first state, the second housing (220) may not be movable relative to the first housing (210) in the second direction (262).

[0074] For example, within the first state, the display (230) may provide the display area having the smallest size. For example, within the first state, the display area may correspond to the first area (230a). For example, although not illustrated in FIG. 2A, within the first state, an area of ​​the display (230) other than the first area (230a) that is the display area (e.g., the second area (230b) of FIG. 2C) may be disposed within the first housing (210). For example, within the first state, the second area (230b) may be covered by the first housing (210). For example, within the first state, the second area (230b) may be moved into the first housing (210). For example, at least a portion of the second area (230b) may be moved (e.g., rolled) into the first housing (210). For example, within the first state, the first region (230a) may include a planar portion. For example, within the first state, a portion of the second region (230b) may include a curved portion. However, this is not limited thereto. For example, the first region (230a) may also include a curved portion extending from the planar portion within the first state.

[0075] For example, the first state may be referred to as a slide-in state in that at least a portion of the second housing (220) is positioned within the first housing (210) as the second housing (220) slides toward the first housing (210). For example, the first state may be referred to as a reduced state in that it provides the display area having the smallest size, but is not limited thereto.

[0076] For example, the second housing (220) may include a front camera (250-1) that obtains visual information through a portion of the first region (230a) and faces a third direction (263) parallel to the z-axis. For example, although not illustrated in FIG. 2A, the second housing (220) may include one or more rear cameras (e.g., rear cameras (250-2) of FIG. 2B) that are visually exposed through a portion of the second housing (220) and face a fourth direction (264) parallel to the z-axis and opposite to the third direction (263). For example, the one or more rear cameras (250-2) may be exemplified with reference to the description of FIG. 2B.

[0077] FIG. 2b is a bottom view of an exemplary electronic device in a first state.

[0078] In FIG. 2B, within the first state, one or more rear cameras (250-2) disposed within the second housing (220) may be positioned within a structure disposed within the first housing (210) for the one or more rear cameras (250-2). For example, since the one or more rear cameras (250-2) are positioned within the structure within the first state, the one or more rear cameras (250-2) may be visually exposed through the structure within the first state. The one or more rear cameras (250-2) may obtain visual information through the structure. For example, the structure may be implemented in various ways. For example, the structure may be an opening or a notch. For example, the structure may be an opening (212a) within a first plate (212) of the first housing (210) that surrounds at least a portion of the second housing (220). However, the present invention is not limited thereto.

[0079] In one or more embodiments, the first state may be changed to the second state.

[0080] For example, the first state (or the second state) can be changed to the second state (or the first state) through one or more intermediate states between the first state and the second state.

[0081] For example, the first state (or the second state) may be changed to the second state (or the first state) based on a defined user input. For example, the first state (or the second state) may be changed to the second state (or the first state) in response to a user input on a physical button visually exposed through a part of the first housing (210) or a part of the second housing (220). There is no limitation on the type of user input. For example, the user input may include a user input through a touch screen within a display area of ​​the display (230) or a user input through a microphone of the electronic device (101). For example, the state of the electronic device (101) may be changed to the second state (or the first state) by an external force applied to the first housing (210) and / or the second housing (220).

[0082] The above second state can be illustrated through the description of FIG. 2c and FIG. 2d.

[0083] FIG. 2c is a plan view of an exemplary electronic device (101) in a second state.

[0084] In FIG. 2C, within the second state, the second housing (220) may be movable relative to the first housing (210) in the second direction (262) among the first direction (261) and the second direction (262). For example, within the second state, the second housing (220) may not be movable relative to the first housing (210) in the first direction (261).

[0085] For example, within the second state, the display (230) may provide the largest display area available in the electronic device (101). For example, within the second state, the display area may correspond to an area (230c) including a first area (230a) and a second area (230b). For example, the second area (230b), which was included within the first housing (210) within the first state, may be visually exposed within the second state. For example, within the second state, the first area (230a) and the second area (230b) may include a planar portion. However, the present invention is not limited thereto. For example, the first area (230a) and / or the second area (230b) may also include a curved portion extending from the planar portion and positioned within the edge portion.

[0086] For example, the second state may be referred to as a slide-out state in that at least a portion of the second housing (220) is positioned outside the first housing (210) according to the second housing (220) sliding from the first housing (210). For example, the second state may be referred to as an expanded state having the largest display area. However, embodiments of the present disclosure are not limited thereto.

[0087] For example, the front camera (250-1) facing the third direction (263) may move together with the first region (230a) in response to the movement of the second housing (220) in the first direction (261) when the state of the electronic device (101) changes from the first state to the second state. For example, although not shown in FIG. 2c, one or more rear cameras facing the fourth direction (264) (e.g., the rear cameras (250-2) of FIG. 2d) may move together with the second housing (220) in response to the movement of the second housing (220) in the first direction (261) when the state of the electronic device (101) changes from the first state to the second state. For example, the relative positional relationship between one or more rear cameras (250-2) and the structure illustrated in the description of FIG. 2B may change according to the movement of one or more rear cameras (250-2). For example, the change in the relative positional relationship may be illustrated in FIG. 2D.

[0088] FIG. 2d is a bottom view of an exemplary electronic device (101) in a second state.

[0089] In FIG. 2d, within the second state, one or more rear cameras (250-2) may be positioned outside the structure. For example, within the second state, one or more rear cameras (250-2) may be positioned outside the opening (212a) in the first plate (212). For example, since one or more rear cameras (250-2) are positioned outside the opening (212a) within the second state, one or more rear cameras (250-2) may be visually exposed within the second state. One or more rear cameras (250-2) positioned outside the structure may acquire visual information. For example, since one or more rear cameras (250-2) are positioned outside the structure within the second state, the relative positional relationship between the one or more rear cameras (250-2) and the structure (e.g., the opening (212a)) within the second state may be different from the relative positional relationship between the one or more rear cameras (250-2) and the structure (e.g., the opening (212a)) within the first state (e.g., FIG. 2b).

[0090] Although not shown in FIGS. 2A, 2B, 2C, and 2D, the electronic device (101) may be in any state (e.g., an intermediate state) between the first state and the second state. For example, the size of the display area in the intermediate state may be larger than the size of the display area in the first state and smaller than the size of the display area in the second state. For example, the display area in the intermediate state may correspond to an area including a portion of the first region (230a) and the second region (230b). For example, in the intermediate state, a portion of the second region (230b) may be visually exposed, and another portion (or a remaining portion) of the second region (230b) may be covered by the first housing (210) or moved into the first housing (210). However, the present invention is not limited thereto.

[0091] The electronic device (101) may include structures for moving a second housing (e.g., the second housing (220) of FIGS. 2A, 2B, 2C, and 2D) of the electronic device (101) relative to a first housing (e.g., the first housing (210) of FIGS. 2A, 2B, 2C, and 2D) of the electronic device (101). While the embodiments describe an electronic device (101) having a flexible display, or multiple housings that can be moved to change a state of the electronic device (101), the embodiments are not limited to such configurations. For example, the electronic device may include a single non-flexible display, and one or more portions of the display may be turned on or off to change the layout size of the display. For example, in a first state, the entire area of ​​the single display may be activated, and in a second state, half of the display may be turned off.

[0092] FIG. 3 is a flowchart illustrating an exemplary operation of an electronic device (101) according to one or more embodiments.

[0093] In FIG. 3, the electronic device (101) of the present disclosure may include a display module including a flexible display that can be expanded (e.g., extended) or contracted (e.g., reduced or retracted). As described above with reference to FIGS. 2A to 2D , the electronic device (101) may be changed into a plurality of states in which the size of the display module is different.

[0094] For example, the state of the display module may include any one of a first state providing a display area having a first size, a second state providing a display area having a second size different from the first size, and an intermediate state providing a display area having a predetermined size between the first size and the second size.

[0095] The electronic device (101) of the present disclosure can efficiently use screen space by optimizing the layout of the home screen, application (app) screen, or lock screen as the display size expands or contracts.

[0096] In one or more embodiments, the electronic device (101) of the present disclosure may perform layout optimization using artificial intelligence ("AI"). The AI ​​model used by the electronic device (101) of the present disclosure may be a single AI model or may be implemented as multiple AI models. The AI ​​model may be composed of a neural network (or artificial neural network) and may include statistical learning algorithms that mimic biological neurons in machine learning and cognitive science. A neural network may refer to a model in general that has problem-solving capabilities by changing the binding strength of synapses through learning, in which artificial neurons (nodes) form a network by combining synapses. The neurons of the neural network may include a combination of weights or biases. The neural network may include one or more layers composed of one or more neurons or nodes. For example, the device may include an input layer, a hidden layer, and an output layer. The neural network constituting the device may infer a desired result (output) from an arbitrary input (input) by changing the weights of neurons through learning.

[0097] In one embodiment, the AI ​​may include generative AI. Generative AI may include a type of AI system capable of generating text, images, or other media in response to input prompts. Generative AI can learn the patterns and structure of input training data and then create new data with similar characteristics.

[0098] At least one processor included in the electronic device (101) can generate a neural network, train or learn a neural network, perform a calculation based on received input data, generate an information signal based on the result of the calculation, or retrain the neural network. The models of the neural network may include various types of models such as CNN (Convolution Neural Network) such as GoogleNet, AlexNet, VGG Network, R-CNN (Region with Convolution Neural Network), RPN (Region Proposal Network), RNN (Recurrent Neural Network), S-DNN (Stacking-based deep Neural Network), S-SDNN (State-Space Dynamic Neural Network), Deconvolution Network, DBN (Deep Belief Network), RBM (Restricted Boltzmann Machine), Fully Convolutional Network, LSTM (Long Short-Term Memory) Network, Classification Network, etc., but are not limited thereto. The processor may include one or more processors for performing operations according to neural network models. For example, the neural network may include a deep neural network. In one or more embodiments, the electronic device (101) may be pre-loaded with one or more pre-trained neural networks. The pre-trained neural networks may be updated based on user data related to the use of the electronic device (e.g., app usage and / or environmental data). In one or more embodiments, the one or more neural networks may be trained remotely and downloaded to the electronic device (101).

[0099] Neural networks include CNN (Convolutional Neural Network), RNN (Recurrent Neural Network), perceptron, multilayer perceptron, FF (Feed Forward), RBF (Radial Basis Network), DFF (Deep Feed Forward), LSTM (Long Short Term Memory), GRU (Gated RecuREnt Unit), AE (Auto Encoder), VAE (Variational Auto) Encoder), DAE (Denoising Auto Encoder), SAE (Sparse Auto Encoder), MC (Markov Chain), HN (Hopfield Network), BM (Boltzmann Machine), RBM (Restricted Boltzmann Machine), DBN (Depp Belief Network), DCN (Deep Convolutional Network), DN (Deconvolutional Network), DCIGN (Deep Convolutional Inverse Graphics Network), Generative Adversarial Network (GAN), Liquid State Machine (LSM), Extreme Learning Machine (ELM), It will be understood by those skilled in the art that any neural network may be included, including but not limited to ESN (Echo State Network), DRN (Deep Residual Network), DNC (Differentiable Neural Computer), NTM (Neural Turning Machine), CN (Capsule Network), KN (Kohonen Network), and AN (Attention Network).

[0100] According to an exemplary embodiment of the present disclosure, at least one processor included in the electronic device (101) is configured to perform a CNN (Convolution Neural Network) such as GoogleNet, AlexNet, and / or VGG Network, R-CNN (Region with Convolution Neural Network), RPN (Region Proposal Network), RNN (Recurrent Neural Network), S-DNN (Stacking-based deep Neural Network), S-SDNN (State-Space Dynamic Neural Network), Deconvolution Network, DBN (Deep Belief Network), RBM (Restricted Boltzmann Machine), Fully Convolutional Network, LSTM (Long Short-Term Memory) Network, Classification Network, Generative Modeling, eXplainable AI, Continual AI, Representation Learning, AI for Material Design, BERT for natural language processing, SP-BERT, MRC / QA, Text Analysis, Dialog System, GPT-3, GPT-4, Visual Analytics for vision processing, Visual Understanding, Video Synthesis, ResNet Anomaly for data intelligence Various AI structures and algorithms can be used, including but not limited to Detection, Prediction, Time-Series Forecasting, Optimization, Recommendation, and Data Creation.

[0101] According to one embodiment, the electronic device (101) of the present disclosure can learn a user's usage pattern according to a usage environment using a first AI model (operation 310), detect a display size based on a change in the state of the display module (operation 320), and perform layout optimization based on the usage pattern and the display size using a second AI model (operation 330).

[0102] According to an example, in operation 310, the electronic device (101) may learn a user's usage pattern according to a usage environment using the first AI model. The electronic device (101) may learn a user's usage pattern based on the usage environment using the first AI model. In one or more embodiments, the usage pattern may correspond to the usage of the electronic device (101), and may include, for example, the frequency of application usage and / or the frequency of a state in which the electronic device (101) is used. For example, the electronic device (101) may track the frequency of application usage, the environment in which the electronic device (101) is used, and / or the frequency of a state in which the electronic device (101) is used.

[0103] In one embodiment, the first AI model may be an analytical AI model. An analytical AI model generally refers to an AI neural network that analyzes user input information and environmental data to learn patterns and make predictions. Analytical AI models may include models that analyze data and derive statistical or rule-based insights. Typical analytical AI models include supervised learning models, unsupervised learning models, and reinforcement learning models. Supervised learning models learn using input data and corresponding labels and can be primarily used to solve classification and regression problems. Unsupervised learning models analyze unlabeled data to discover hidden patterns or structures and can be primarily used for clustering and dimensionality reduction. Reinforcement learning models are models in which an agent learns to maximize rewards while interacting with an environment, and can be primarily used to learn optimal action policies.

[0104] In one embodiment, the first AI model may be a generative AI model. A generative AI model may refer to an AI neural network that generates new types of data based on user input information. Generative AI models can generate various types of data, such as image generation, text generation, and music generation. Representative generative AI models include a Generative Adversarial Network (GAN), a Variational Autoencoder (VAE), and a Diffusion-based model. The first AI model, as a generative AI model, may perform learning to analyze or tune input data. For example, the first AI model may learn a user's app (or application) usage pattern based on the input data. For example, the first AI model may continuously analyze the input data to provide a customized interface tailored to the user environment. In one embodiment, the electronic device (101) may analyze the usage environment of the electronic device using the first AI model. For example, the usage environment may include at least one of display size, location, time, movement speed, network status, remaining battery level, weather, ambient illumination, ambient noise, and ambient temperature.

[0105] For example, the electronic device (101) can determine or identify whether the display size has been expanded (e.g., increased) or reduced (e.g., decreased). In one or more embodiments, the electronic device (101) may include a sensor that detects whether the electronic device (101) has been expanded or contracted. The electronic device (101) can determine whether the location where the electronic device (101) is used is indoors or outdoors. For example, the electronic device (101) can determine whether the location where the electronic device (101) is used is home or office. The electronic device (101) can determine whether the time of day when the electronic device (101) is used is morning, afternoon, or evening. The electronic device (101) can determine whether the user of the electronic device (101) is stationary, walking, or driving based on the movement speed. The electronic device (101) can determine whether a battery saving mode is required by analyzing the remaining battery level. The electronic device (101) can determine the current weather based on weather data and location information. The electronic device (101) can determine the brightness of the environment in which the electronic device (101) is used by sensing the ambient illuminance. The electronic device (101) can determine the sound of the environment in which the electronic device (101) is used by sensing the ambient noise. The electronic device (101) can determine the possibility of overheating of the electronic device (101) by sensing the ambient temperature. In addition, the electronic device (101) can predict the current state of the user by analyzing the usage environment. In addition, the electronic device (101) can track which applications are used when the electronic device (101) is in an expanded or contracted state and / or when the electronic device (101) is used in a specific environment.

[0106] According to one embodiment, the electronic device (101) can learn a user's usage pattern using the first AI model. For example, the usage pattern may include at least one of the type of app (or application) used, the frequency of app use, the duration of app use, the type of widget used, the frequency of widget use, the duration of widget use, the wallpaper setting, the screen brightness setting, or the resolution setting according to the usage environment.

[0107] In one embodiment, a user's usage pattern may change depending on the usage environment. The electronic device (101) learns the user's usage pattern based on the usage environment using the first AI model, thereby predicting the user's usage pattern based on a given usage environment.

[0108] For example, the electronic device (101) can learn the types of frequently used apps based on the usage environment. For example, the electronic device (101) can learn the usage time or frequency of use of a specific app (or application) based on the usage environment. For example, the electronic device (101) can learn the types of frequently used widgets based on the usage environment. For example, the electronic device (101) can learn the usage time or frequency of use of a specific widget based on the usage environment. For example, the electronic device (101) can learn the method of changing the wallpaper based on the usage environment. For example, the electronic device (101) can learn the screen brightness setting based on the usage environment. For example, the electronic device (101) can learn the resolution setting based on the usage environment.

[0109] In one example, at operation 320, the electronic device (101) can detect a display size based on a change in the state of the display module. For example, the electronic device (101) can detect a change in the state of the display module and calculate a display size using at least one sensor and a software algorithm. The display module can provide different display sizes corresponding to various states. For example, the electronic device (101) can calculate a change in the size of the display area of ​​the display module.

[0110] According to one embodiment, the electronic device (101) can change the size of a display area for displaying an image to a user based on a change in the state of the display module (e.g., folding, unfolding, sliding in, sliding out). For example, the operation of calculating the display size may include an operation of calculating a change in the size of the display area of ​​the display module.

[0111] For example, the electronic device (101) can detect the display size using at least one of position sensors, length measurement sensors, magnetic sensors, or potentiometric sensors.

[0112] For example, the electronic device (101) can calculate the changed display size by measuring the physical change of the display module using an image processing software algorithm.

[0113] In one example, at operation 330, the electronic device (101) may perform layout optimization based on the usage pattern and the display size using the second artificial intelligence model.

[0114] In one embodiment, the second AI model may be a generative AI model. A generative AI model generally refers to an AI neural network that generates new types of data based on user input information. Generative AI models may include models that generate images and / or models that generate language. Representative models for generating images include generative adversarial networks (GANs) and variational autoencoders (VAEs), and examples include diffusion-based generative models that use VAEs and Transformer structures. Language generating models are models trained to statistically produce the most appropriate output based on input values, and representative examples include models such as CHAT-GPT 3 and CHAT-GPT 4. There are also large multimodal models (LMMs) that can recognize various types of data input, such as text, images, and voice, and generate new data corresponding to them.

[0115] According to one embodiment, the electronic device (101) may perform layout optimization on at least one screen among a home screen, an app screen, or a lock screen based on the usage pattern and the display size. Hereinafter, the operation of the electronic device performing layout optimization will be described in detail with reference to FIGS. 4A to 6.

[0116] FIGS. 4A and 4B are exemplary diagrams illustrating layout optimization performed on a home screen of an electronic device (400) according to one or more embodiments.

[0117] The electronic device (400) may perform the layout optimization including at least one of: arranging app icons (410) on the home screen, resizing app icons (410), arranging widgets (430), changing the configuration of widgets (430), setting app foldering (420), changing a wall page (440) image, or editing a status bar (450). In one or more embodiments, the arranging of app icons and / or widgets may include rearranging the layout of app icons and / or widgets in response to an expansion or contraction of the display screen size. In one or more embodiments, the sizing of app icons and / or widgets may include resizing one or more app icons and / or widgets in response to an expansion or contraction of the display screen size. In one or more embodiments, the app foldering setting may include organizing one or more app icons and / or widgets into existing or new folders.

[0118] For example, as shown in FIG. 4A, when the display size is expanded (e.g., increased), the electronic device (400) may perform at least one of adding an app icon (410), enlarging the size of an app icon (410), adding a type of widget (430), splitting a widget (430), enlarging the size of a widget (430), releasing app foldering (420), expanding a wallpaper (440) image, or adding a status bar (450) based on the expanded display size (e.g., the size of the increased display area in the display module).

[0119] For example, as shown in FIG. 4B, when the display size is reduced (e.g., decreased), the electronic device (400) may perform at least one of deleting an app icon (410), reducing the size of an app icon (410), deleting a widget type, integrating widgets, reducing the size of widgets, app foldering (420), reducing a wallpaper image, or deleting a status bar (450) based on the reduced display size (e.g., the size of a reduced display area in a display module). For example, when the display size is reduced, the electronic device (400) may delete at least one of the first widget (430a) and the second widget (430b) based on the reduced display size. For example, when the display size is reduced, the electronic device (400) may integrate the first widget (430a) and the second widget (430b) into one widget based on the reduced display size.

[0120] FIGS. 5A to 5C are exemplary diagrams illustrating layout optimization performed on an app screen of an electronic device (500) according to one or more embodiments.

[0121] In FIGS. 5A to 5C, the electronic device (500) can perform layout optimization corresponding to the display size based on the app's unique functions on the app screen.

[0122] In one embodiment, when the display size is expanded (e.g., increased), the electronic device (500) may provide additional information that complements at least one content on the app screen. For example, as shown in FIG. 5A, when changing from a first state to a second state, the electronic device (500) may additionally display content (501) of a running app in the expanded area (EA). For example, as shown in FIG. 5B, when changing from a first state to a second state, the electronic device (500) may display at least one other app (502) in the expanded area (EA). For example, the electronic device (500) may display another recently executed app (502a) or another app (502b) that the user is expected to execute. For example, as shown in FIG. 5C, when changing from a first state to a second state, the electronic device (500) may execute a multi-window and display at least one other app (503) in the expanded area (EA).

[0123] In one embodiment, the electronic device (500) may provide summary information on compressing (e.g., reducing) at least one piece of content on the app screen when the display size is reduced (e.g., reduced). For example, when changing from a second state to a first state, the electronic device (500) may display content with a reduced amount of content based on the priority of the content. For example, when changing from a second state to a first state, the electronic device (500) may display compressed content based on the priority of the content.

[0124] FIG. 6 is an exemplary diagram illustrating layout optimization performed on a lock screen of an electronic device (600) according to one or more embodiments.

[0125] In FIG. 6, the electronic device (600) can perform layout optimization corresponding to the display size based on the notification importance on the lock screen.

[0126] In one embodiment, when the display size is expanded, the electronic device (600) may provide additional information that complements at least one content on the lock screen. For example, as shown in FIG. 6, when changing from a first state to an intermediate state, the electronic device (600) may display notification contents (602a, 602b, 603c) (EA1) corresponding to the notification icon (601) in the expanded area. For example, as shown in FIG. 6, when changing from an intermediate state to a second state, the electronic device (600) may display one or more additional notification contents (603) (EA2) corresponding to the expanded display size. The notification contents may correspond to contents regarding recently received messages (e.g., text messages, phone calls, etc.) and / or recent app notifications. The notification contents may correspond to audio (e.g., podcast) playing in the background.

[0127] In one embodiment, the electronic device (600) may provide summary information that compresses at least one of the contents on the lock screen when the display size is reduced. For example, when changing from a second state to an intermediate state, the electronic device (600) may remove notifications (e.g., additional notifications (603)) in descending order of importance based on the reduced display size. For example, when changing from an intermediate state to a first state, the electronic device (600) may display a compressed notification by replacing the notification contents (602a, 602b, 603c) with a notification icon (601).

[0128] FIG. 7 is an exemplary diagram illustrating layout optimization for an app icon (710) of an electronic device according to one or more embodiments.

[0129] In FIG. 7, the electronic device can adjust the size of the app icon (710). For example, when the display size is expanded, the electronic device can increase the size of the app icon (710) on the home screen to improve readability or touch accuracy. For example, when the display size is expanded from a standard screen ratio of 90% to a standard screen ratio of 100%, the size of the app icon (710) can increase from 90% of the standard app size to 100% of the standard app size in response to the change in the display size. For example, when the display size is reduced, the electronic device can reduce the size of the app icon (710) to place more icons on the screen. For example, when the display size is reduced from a standard screen ratio of 100% to a standard screen ratio of 90%, the size of the app icon (710) can decrease from 100% of the standard app size to 90% of the standard app size in response to the change in the display size. In one or more embodiments, instead of resizing the app icons, one or more app icons may be removed from the screen while maintaining the size of the remaining icons.

[0130] In one embodiment, the electronic device can adjust the arrangement of app icons (710). For example, the electronic device can place app icons (710) that the user frequently uses in a predetermined location on the home screen (e.g., at the top or bottom of the home screen). For example, when the display size is expanded, the electronic device can place app icons (710) in order of the user's most frequently used apps. For example, when the display size is reduced, the electronic device can remove app icons (710) in order of the user's least frequently used apps.

[0131] FIG. 8 is an exemplary diagram illustrating layout optimization for app foldering (820) of an electronic device according to one or more embodiments. App foldering may refer to selecting and grouping app icons into specific folders. For example, app icons may be grouped according to similar functions or categories. For example, by performing app foldering (820), the electronic device may organize app icons on the home screen or app screen.

[0132] In FIG. 8, the electronic device can folder (820) app icons (810) or unfold (820) app icons (810) based on the display size. For example, when the display size is reduced, the electronic device can folder (820) app icons (810). For example, when the display size is expanded, the electronic device can unfold (820) app icons (810). In this way, the electronic device can use the home screen or app screen by automatically performing app folders based on the display size. For example, the electronic device can save screen space by automatically performing app folders when the display is reduced. For example, the electronic device can improve accessibility to important app icons by grouping similar app icons based on the user's usage pattern. In one or more embodiments, when the electronic device changes from a reduced state to an expanded state, the app folders can be reversed so that apps grouped in folders can be unfolded from the folders.

[0133] Electronic devices can analyze a user's app usage patterns and automatically folder (820) multiple apps based on frequency of use and time of day. For example, the electronic device can create an entertainment folder by foldering (820) multiple apps that the user primarily uses during their leisure time.

[0134] Electronic devices can analyze the functions of apps frequently used by users and automatically folder (820) apps with similar functions. For example, the electronic device can create a video viewing folder by foldering (820) apps that display images, such as video playback apps, gallery apps, streaming apps, and OTT apps.

[0135] FIG. 9 is an exemplary diagram illustrating layout optimization for a widget (930) by an electronic device according to one or more embodiments.

[0136] In FIG. 9, the electronic device can adjust at least one of widget placement, widget configuration, widget type, and widget size.

[0137] In one embodiment, the electronic device may determine the placement of widgets based on the user's usage patterns when the display size is expanded or reduced. For example, the electronic device may place the widget (930) in a key location on the home screen based on the user's frequency of use of the widget (930). The widget (930) may be frequently used by the user. For example, the electronic device may place the weather widget and calendar widget on the top portion of the home screen.

[0138] In one embodiment, the electronic device can change the configuration of the widget (930) based on the display size. For example, the electronic device can determine the configuration of the widget based on (e.g., based on) the user's usage pattern when the display size is expanded or reduced. For example, when the display size is reduced, the electronic device can integrate the first widget (930a) and the second widget (930b) that the user frequently checks (e.g., uses) into a single widget (930c). For example, when the display size is expanded, the electronic device can separate the integrated widget (930c) into separate first widget (930a) and second widget (930b), respectively.

[0139] In one embodiment, the electronic device can change the widget type based on the display size. For example, the electronic device can determine the widget type based on (e.g., based on) the user's usage pattern when the display size is expanded or reduced. For example, when the display size is expanded, the electronic device can add a frequently used widget (930) type. For example, when the display size is reduced, the electronic device can remove a widget (930) type with a low frequency of use.

[0140] In one embodiment, the electronic device can change the widget size based on the display size. For example, the electronic device can determine the widget size based on (e.g., based on) the user's usage pattern when the display size is expanded or reduced. For example, when the display size is expanded, the electronic device can increase the size of the widget (930) corresponding to the expanded ratio. For example, when the display size is reduced, the electronic device can decrease the size of the widget (930) corresponding to the reduced ratio.

[0141] FIGS. 10A and 10B are exemplary diagrams illustrating layout optimization for a wall paper (1040) of an electronic device according to one or more embodiments.

[0142] In FIGS. 10A and 10B, the electronic device can adjust the image of the wall wallpaper (e.g., 1040a, 1040d). For example, the electronic device can expand or contract the image of the wall wallpaper (e.g., 1040a, 1040d) based on the display size.

[0143] As shown in FIG. 10A, when the display size is expanded, the electronic device can expand the wall wallpaper (1040a) image by generating an image corresponding to the expanded area. For example, when changing from the first state to the intermediate state, the electronic device can generate a wallpaper (1040b) including an additional image for the expanded area from the wall wallpaper (1040a) image using a second AI model (e.g., a generative AI model). For example, when changing from the intermediate state to the second state, the electronic device can generate a wallpaper (1040c) including an additional image for the expanded area from the wall wallpaper (1040b) image using the second AI model.

[0144] According to one embodiment, the electronic device may generate a prompt (e.g., a message or notification) for outputting an expanded wallpaper image based on image information for the wallpaper (1040a) and expanded display size information. The electronic device may transmit an input value corresponding to an intermediate state or a second state to the second AI model based on the generated prompt. The electronic device may obtain and display an expanded wallpaper image suitable for each display size as an output value of the second AI model. For example, the electronic device may provide the current wallpaper image data and the size information of the expanded display to the second AI model as input values, and generate a prompt saying "Expand the current wallpaper to generate an image suitable for the new screen size." The second AI model may generate an additional image corresponding to the expanded display based on the prompt, and output a wallpaper image reflecting the additional image.

[0145] As shown in FIG. 10b, when the display size is reduced, the electronic device can analyze the wallpaper (1040d) image, divide the wallpaper (1040d) image into predetermined sub-regions, and determine the importance of each sub-region. The electronic device can remove the sub-regions from the wallpaper (1040d) image in descending order of importance. For example, if the electronic device determines that the importance of the first sub-region (e.g., the sea) is high, the electronic device can reduce the wallpaper (1040e) image centered on the first sub-region (e.g., a partial image or a specific region). For example, if the electronic device determines that the importance of the second sub-region (e.g., a tree) is high, the electronic device can reduce the wallpaper (1040f) image centered on the second sub-region (e.g., a partial image or a specific region).

[0146] According to one embodiment, the electronic device may generate a prompt for outputting a reduced wallpaper image based on image information for the wallpaper (1040d) and information on the reduced display size. The electronic device may transmit an input value corresponding to the reduced state to the second AI model based on the generated prompt. The electronic device may obtain and display a reduced wallpaper image suitable for each display size as an output value of the second AI model. For example, the electronic device may provide the current wallpaper image data and the size information of the reduced display to the second AI model as input values, and generate a prompt saying "Reduce the current wallpaper to generate an image suitable for the new screen size." The second AI model may generate an optimized image corresponding to the reduced display based on the prompt, and output a wallpaper image reflecting the optimized image.

[0147] FIG. 11 is an exemplary diagram illustrating layout optimization in consideration of a wall paper (1140) image by an electronic device according to one or more embodiments.

[0148] In FIG. 11, the electronic device can adjust the placement of app icons (1110) or widgets based on the wallpaper image (1140a).

[0149] The electronic device may position the app icon (1110) or widget so that a region of interest (ROI) (e.g., a person's face or a specific point) is not obscured in the wallpaper image (1140a). For example, the electronic device may position the app icon (1110) or widget so as to avoid the region of interest (ROI) in the wallpaper image (1140a).

[0150] Additionally, when the wallpaper image (1140a) is changed to the wallpaper image (1140b), the electronic device can re-evaluate the region of interest (ROI) of the wallpaper image (1140b) and adaptively adjust the placement of the app icon (1110) or widget based on the region of interest (ROI).

[0151] In one embodiment, the electronic device may analyze a wallpaper image (1140a) using a first AI model (e.g., an analytical AI model). The first AI model may analyze the input wallpaper image (1140a) to identify a region of interest (ROI) and determine an optimal layout based on the ROI. For example, the electronic device may analyze various features of the wallpaper image (1140a) to assess the importance of a person's face, specific points, and / or background elements. The first AI model may determine the placement of app icons (1110) and widgets based on the analysis results of the wallpaper image (1140a).

[0152] In one or more embodiments, the first AI model may use computer vision technology to identify a region of interest (ROI) in the wallpaper image (1140a). For example, the first AI model may detect a person's face using a facial recognition algorithm or identify important background elements using a point detection algorithm. For example, based on the identification of the region of interest (ROI), the electronic device may generate an optimal layout in which app icons (1110) and widgets are arranged to avoid the region of interest. In one or more embodiments, the region of interest (ROI) may be determined by performing edge detection. For example, the edge detection process may be performed to determine one or more edges of an object, and an area including the one or more edges may be identified as a region of interest (ROI). The app icon or widget may be positioned so as not to overlap the detected edges.

[0153] Additionally, when the wallpaper image (1140a) is changed to a wallpaper image (1140b), the first AI model can reanalyze the new wallpaper image (1140b) to identify a new region of interest (ROI). The electronic device can readjust the positions of the app icon (1110) and widgets in response to the new wallpaper image (1140b).

[0154] FIG. 12 is an exemplary diagram illustrating layout optimization for a status bar (1250) of an electronic device (1200) according to one or more embodiments.

[0155] In FIG. 12, the electronic device (1200) can adjust the contents of the status bar (1250) based on the display size.

[0156] The electronic device (1200) can adjust the icon size and spacing of the status bar (1250) based on the display size. For example, the electronic device (1200) can increase the icon size and spacing of the status bar (1250) when the display size is expanded. For example, the electronic device (1200) can decrease the icon size and spacing of the status bar (1250) when the display size is reduced.

[0157] The electronic device (1200) can change the configuration displayed on the status bar (1250) based on the display size. For example, when the display size is expanded, the electronic device (1200) can add a number of icons corresponding to the expanded area. For example, when the display size is reduced, the electronic device (1200) can remove icons from the status bar (1250) in descending order of frequency of use.

[0158] According to one embodiment, the electronic device (1200) may analyze the usage status and display size using a first AI model (e.g., an AI artificial intelligence model), and optimize the layout of the status bar based on the usage status and display size. For example, the first AI model may learn the user's app usage pattern, frequency, importance, etc., and determine the optimal status bar layout corresponding to the usage status. For example, the first AI model may analyze the apps or functions frequently used by the user, and adjust the arrangement and size of status bar icons based on changes in the display size.

[0159] In the second state, the electronic device (1200) may configure the status bar around frequently used icons and maintain the icon size and spacing at the default size. For example, in the second state, the status bar (1250) may include at least one of detailed time, Wi-Fi status, signal strength, Bluetooth connection status, and battery. For example, in the second state, the app screen may include a first app icon (1211) and a second app icon (1212).

[0160] In the intermediate state, the electronic device (1200) may reduce the configuration of the status bar and app screen in response to the reduced screen size, taking into account the user's app usage pattern, frequency, and importance. For example, in the intermediate state, the status bar (1251) may include at least one of a simplified time, Wi-Fi status, and battery. For example, in the intermediate state, the app screen may include only the first app icon (1211) without the second app icon (1212).

[0161] In the first state, the electronic device (1200) may reduce (e.g., decrease or reduce) the configuration of the status bar and app screen in response to the minimized screen, taking into account the user's app usage pattern, frequency, and importance. For example, in the first state, the status bars (1250, 1251) may be removed. For example, in the first state, both the first app icon (1211) and the second app icon (1212) may be removed from the app screen.

[0162] FIG. 13 is a flowchart illustrating an exemplary operation of an electronic device according to one or more embodiments to perform layout optimization based on a state change intent.

[0163] In FIG. 13, the electronic device of the present disclosure can determine a user's intention to change a state based on a usage environment and the display size (operation 1310), predict the user's next action based on the intention to change the state (operation 1320), and perform layout optimization based on the predicted next action (operation 1330).

[0164] For example, in operation 1310, the electronic device may determine the user's intention to change state based on the usage environment and display size.

[0165] Electronic devices can determine a user's current status or activity by analyzing the user's usage environment in real time. When the display module's status changes, the electronic device can determine the intent to change the status based on the usage environment or the type of app being run. For example, when a call app is in use and the display size expands, the electronic device can determine the intent to change the status to start a call. For example, when a navigation app is in use and the display size shrinks (e.g., decreases), the electronic device can determine the intent to change the status to start driving.

[0166] In one example, at operation 1320, the electronic device can predict the user's next action based on the state change intention.

[0167] Electronic devices can use a second AI model (e.g., a generative AI model) to predict the user's likely actions in the current situation based on the user's past behavior patterns. For example, the electronic device may predict the user's next action as performing a specific activity. For example, the electronic device may predict the user's next action as executing a specific app or function.

[0168] In one example, at operation 1330, the electronic device may perform the layout optimization based on the predicted next action.

[0169] Electronic devices can optimize the layout of their home screens, app screens, lock screens, and other elements based on the predicted next user action. For example, the device can optimize the placement of apps, widgets, settings, and other elements likely to be needed by the user based on the predicted next action. For example, the device can optimize the layout by adjusting icon size, placement, widget configuration, and other elements based on the display size, enabling the user to efficiently use the device when the predicted next action is performed.

[0170] FIG. 14 is an exemplary diagram illustrating layout optimization performed by an electronic device (1400) while running a music app according to one or more embodiments.

[0171] In FIG. 14, the electronic device (1400) can detect a state change in which the display expands while the user is running a music app. The state of the display module of the electronic device (1400) can change from a first state to a second state. For example, the electronic device (1400) can determine that the user is listening to music through the music app by analyzing the usage environment.

[0172] The electronic device (1400) may predict that when the screen is expanded while the music app is running, the user is likely to view additional information about the music. As the display size expands, the electronic device (1400) may add a lyrics window (AL) for the music to the expanded area of ​​the music app.

[0173] According to one embodiment, the electronic device (1400) may use a first AI model (e.g., an analytical AI model) to collect and analyze various environmental data, such as the user's location, time zone, network status, battery status, and ambient noise, to learn the user's primary usage patterns for the music app. For example, the electronic device (1400) may identify the time zone or location where the user listens to music and analyze the user's usage patterns for the music app.

[0174] According to one embodiment, the electronic device (1400) may perform layout optimization based on the user's usage pattern and display size using a second AI model (e.g., a generative AI model). For example, the electronic device (1400) may perform layout optimization to add a lyrics window when the screen is expanded while the user is running a music app based on usage pattern data learned from the first AI model. Specifically, the electronic device (1400) may use the user's current music app usage pattern and display expansion information as input values ​​for the second AI model. For example, the second AI model may determine the optimal layout based on a prompt such as "When the user is listening to music and the display is expanded, provide additional information." For example, in response to the display expansion, the electronic device (1400) may add a lyrics window to the music app interface and provide the expanded information to the user.

[0175] FIG. 15 is an exemplary diagram illustrating layout optimization performed by an electronic device (1500) while running a call app according to one or more embodiments.

[0176] In FIG. 15, the electronic device (1500) can detect a state change in which the display expands while the user is running a call app. The state of the display module of the electronic device (1500) can change from a first state to a second state. For example, the electronic device (1500) can determine that the user is on a call by analyzing the usage environment.

[0177] The electronic device (1500) may predict that when the screen of a call app is expanded while the call app is running, the user is more likely to view additional information about the call. As the display size expands, the electronic device (1500) may add a conversation summary window (AL) to provide a summary of the conversation in the expanded area of ​​the call app. As the display size expands, the electronic device (1500) may add a note-taking window (AL) to the expanded area of ​​the call app, allowing the user to take notes during the call.

[0178] According to one embodiment, the electronic device (1500) may use a first artificial intelligence model (e.g., an analytical artificial intelligence model) to collect and analyze various environmental data, such as the user's location, time zone, network status, battery status, and ambient noise, to learn the user's primary usage patterns for a calling app. For example, the electronic device (1500) may identify the time zones and locations where the user primarily makes calls and analyze the user's usage patterns for the calling app.

[0179] According to one embodiment, the electronic device may perform layout optimization based on the user's usage pattern and display size using a second AI model (e.g., a generative AI model). For example, the electronic device (1500) may perform layout optimization by adding a conversation summary window when the screen is expanded while the call app is running, based on usage pattern data learned from the first AI model. Specifically, the electronic device (1500) may use the current user's usage pattern of the call app and display expansion information as input values ​​for the second AI model. For example, the second AI model may determine the optimal layout based on a prompt such as "When the user is on a call and the display is expanded, provide additional information." For example, in response to the display expansion, the electronic device (1500) may add a conversation summary window and a memo window to the call app interface and provide the expanded information to the user.

[0180] FIG. 16 is an exemplary diagram illustrating layout optimization performed by an electronic device (1600) while executing an Internet app according to one or more embodiments.

[0181] In FIG. 16, the electronic device (1600) can detect a state change in which the display is reduced while the user is running an Internet app. The state of the display module of the electronic device (1600) can change from a second state to a first state. For example, the electronic device (1600) can determine that the user is reading an article through an Internet app by analyzing the usage environment.

[0182] When the screen size of an Internet app is reduced while the electronic device (1600) is running, the electronic device (1600) can anticipate that the user intends to use the screen space as efficiently as possible. As the display size is reduced, the electronic device (1600) can remove the address bar (1610) from the Internet app and display a summary of the article content (1620). As the display size is reduced, the electronic device (1600) can reduce the size of the image (1630) in the Internet app and remove advertisements (1640).

[0183] According to one embodiment, the electronic device (1600) may use a first AI model (e.g., an analytical AI model) to collect and analyze various environmental data, such as the user's location, time zone, network status, battery status, and ambient noise, to learn the user's primary Internet app usage patterns. For example, the electronic device (1600) may identify the time zone or location where the user primarily reads articles and analyze the user's Internet app usage characteristics.

[0184] According to one embodiment, the electronic device can perform layout optimization based on the user's usage pattern and display size using a second AI model (e.g., a generative AI model). For example, based on usage pattern data learned from the first AI model, the electronic device (1600) can perform layout optimizations, such as removing the address bar (1610) and displaying an article summary (AL), when the screen is reduced while running an Internet app. Specifically, the electronic device (1600) can use the user's current Internet app usage pattern and display reduction information as input values ​​for the second AI model. For example, the second AI model can determine the optimal layout based on a prompt such as "When the user is reading an article and the display is reduced, optimize the layout to use the screen space as efficiently as possible." For example, in response to the display reduction, the electronic device (1600) can remove the address bar from the Internet app interface, display a summary of the article, reduce the image size, and remove advertisements to provide an optimized screen for the user.

[0185] FIGS. 17A and 17B are exemplary diagrams illustrating layout optimization performed by an electronic device (1700) while running a navigation app according to one or more embodiments.

[0186] In FIG. 17A, the electronic device (1700) can detect a state change in which the display is reduced (e.g., decreased) while the user is running a navigation app. The state of the display module of the electronic device (1700) can change from a second state to a first state. For example, the electronic device (1700) can determine that the user requires route guidance via the navigation app by analyzing the usage environment.

[0187] The electronic device (1700) may anticipate that the user will likely require improved accessibility to route selection options when the screen is reduced while running a navigation app. As the display size is reduced, the electronic device (1700) may display route selection options in the form of intuitive icons.

[0188] According to one embodiment, the electronic device (1700) may use a first AI model (e.g., an analytical AI model) to collect and analyze various environmental data, such as the user's location, time zone, network status, battery status, and ambient noise, to learn the user's primary usage patterns of the navigation app. For example, the electronic device (1700) may identify the time zones or locations where the user requires route guidance and analyze the user's navigation app usage characteristics.

[0189] According to one embodiment, the electronic device may perform layout optimization based on the user's usage pattern and display size using a second AI model (e.g., a generative AI model). For example, the electronic device (1700) may perform layout optimization to display route selection options in the form of intuitive icons when the screen is reduced while running a navigation app based on usage pattern data learned from the first AI model. Specifically, the electronic device (1700) may use the current user's navigation app usage pattern and display reduction information as input values ​​for the second AI model. For example, the second AI model may determine the optimal layout based on a prompt such as "When the user requires route guidance and the display is reduced, provide route selection options in the form of intuitive icons." For example, in response to the display reduction, the electronic device (1700) may add route selection options in the form of intuitive icons to the navigation app interface, thereby providing the user with improved accessibility to the options.

[0190] In FIG. 17b, the electronic device (1700) can detect a state change in which the display expands while the user is running a navigation app. The state of the display module of the electronic device (1700) can change from a first state to a second state. For example, the electronic device (1700) can determine that the user is driving by analyzing the usage environment.

[0191] The electronic device (1700) may predict that the user intends to use another app simultaneously with the navigation app when the screen is expanded while the navigation app is running. As the display size expands, the electronic device (1700) may add (AL) an app (e.g., a music app) that is frequently used simultaneously with the navigation app to the expanded area of ​​the navigation app.

[0192] According to one embodiment, the electronic device (1700) may use a first artificial intelligence model (e.g., an analytical artificial intelligence model) to collect and analyze various environmental data, such as the user's location, time zone, network status, battery status, and ambient noise, to learn the user's pattern of primarily using a navigation app. For example, the electronic device (1700) may identify the time zone or location at which the user primarily uses the navigation app while driving, and analyze the user's navigation app usage characteristics.

[0193] According to one embodiment, the electronic device may perform layout optimization based on the user's usage pattern and display size using a second AI model (e.g., a generative AI model). For example, the electronic device (1700) may perform layout optimization by adding a music app when the screen is expanded while running a navigation app based on usage pattern data learned from the first AI model. Specifically, the electronic device (1700) may use the current user's navigation app usage pattern and display expansion information as input values ​​for the second AI model. For example, the second AI model may determine the optimal layout based on a prompt such as "When the user is driving and the display is expanded, provide an additional app." For example, in response to the display expansion, the electronic device (1700) may add a music app to the navigation app's interface and provide the user with the ability to control music while playing the navigation.

[0194] FIG. 18 is an exemplary diagram illustrating a settings window (1810) for layout optimization of an electronic device (1800) according to one or more embodiments.

[0195] In FIG. 18, the electronic device (1800) may provide a settings window (1810) for setting (e.g., configuring) whether to apply layout optimization. For example, a user may use the settings window (1810) to activate (e.g., turn on) the layout optimization mode or deactivate (e.g., turn off) the layout optimization mode.

[0196] For example, the settings window (1810) may be provided on the settings app. For example, the settings window (1810) may be provided as a quick settings menu in the status bar. For example, the settings window (1810) may be displayed in a pop-up window format.

[0197] FIG. 19 is a flowchart illustrating an operation of an electronic device according to one or more embodiments to display a layout optimization execution queue.

[0198] In FIG. 19, the electronic device of the present disclosure can display at least one layout optimization execution queue indicating the execution of the layout optimization through the display module. The electronic device can display at least one layout optimization execution queue indicating the execution of the layout optimization through the display module.

[0199] According to one embodiment, the electronic device may learn a user's usage pattern based on a usage environment using a first AI model (operation 1910), detect a display size based on a change in the state of the display module (operation 1920), display at least one execution queue that notifies the user of layout optimization (operation 1930), and perform layout optimization based on the usage pattern and the display size using a second AI model (operation 1940).

[0200] For example, in operation 1910, the electronic device may learn the user's usage pattern based on the usage environment using the first AI model.

[0201] An electronic device can analyze its usage environment using the first AI model. For example, the usage environment may include at least one of display size, location, time, movement speed, network status, battery level, weather, ambient lighting, ambient noise, or ambient temperature.

[0202] Electronic devices can learn a user's usage patterns using the first AI model. For example, the usage patterns may include at least one of the following: type of app used, frequency of app use, duration of app use, type of widget used, frequency of widget use, duration of widget use, wallpaper settings, screen brightness settings, or resolution settings, depending on the usage environment.

[0203] For example, in operation 1920, the electronic device may detect a display size based on a change in the state of the display module. For example, the electronic device may detect a change in the display size using at least one sensor and a software algorithm, and calculate the display size.

[0204] According to one embodiment, the display module may include a sliderable display. The sliderable display may have a structure in which the display module expands or contracts. For example, the sliderable display may change the display area by expanding or contracting the display side through a sliding mechanism. For example, the electronic device may have a screen that expands (e.g., expands) and the display area enlarged as the display module slides out. For example, the electronic device may have a screen that contracts and the display area reduced as the display module slides in.

[0205] When the display module slides out, the electronic device can provide additional information to the user by placing more widgets and icons on the display. For example, when using a music app, if the display module slides out, the electronic device can add a lyrics window to provide additional lyric information.

[0206] When the display module slides in, the electronic device can improve display efficiency by removing unnecessary icons from the display or performing app folders. For example, when using a navigation app, the electronic device can display route selection options in the form of icons when the display module slides in, thereby improving accessibility.

[0207] According to one embodiment, the display module may include a foldable display. The foldable display may have a structure in which the display module can be folded or unfolded. For example, the foldable display may have a display that is divided into at least two parts and folded, thereby changing the display area. For example, the electronic device may have a screen that expands (e.g., increases) and the display area increases as the display module unfolds. For example, the electronic device may have a screen that shrinks and the display area decreases as the display module folds.

[0208] When the display module is unfolded, the electronic device can provide additional information to the user by placing more widgets and icons on the display. For example, when using a calling app, the electronic device can add a conversation summary window and a memo window when the display module is unfolded to provide the user with necessary and / or additional information.

[0209] When the display module folds, electronic devices can improve display efficiency by removing unnecessary icons from the display or performing app folders. For example, when using an internet app, the electronic device can efficiently utilize screen space by removing the address bar and displaying a summary of the article content when the display module folds.

[0210] According to one embodiment, the electronic device can calculate a change in the size of a display area of ​​the display module based on a change in the state of the display module (e.g., folding, unfolding, slide-in, and slide-out).

[0211] For example, in operation 1930, the electronic device (2000) may display at least one layout optimization execution queue indicating the performance of layout optimization through the display module. Hereinafter, operation 1930 of the electronic device (2000) will be described in detail with reference to FIGS. 20A to 20D.

[0212] FIGS. 20A to 20D are exemplary diagrams illustrating a layout optimization execution queue of an electronic device (2000) according to one or more embodiments.

[0213] In FIGS. 20A to 20D , the electronic device (2000) may provide various visual effects to notify the user of the performance (e.g., execution) of layout optimization through the display module. The electronic device (2000) may provide the user with a notification of layout optimization execution by displaying a layout optimization execution queue. For example, the electronic device (2000) may display at least one layout optimization execution queue among a border effect, a blinking effect, an effect of the screen expanding and then returning, or a message notification.

[0214] For example, as shown in FIG. 20A, when performing or executing layout optimization, the electronic device (2000) may display a border effect (2010) in which the border of the display module is highlighted in a specific color. The border effect (2010) may be illuminated for a predetermined period of time and may disappear as the layout optimization is completed.

[0215] For example, as shown in FIG. 20b, the electronic device (2000) may display a blinking effect (2020) in which the display module temporarily blinks when performing layout optimization. The blinking effect (2020) may be repeatedly turned on and off as layout optimization is completed.

[0216] For example, as shown in FIG. 20c, the electronic device (2000) may display an effect (2030) in which the screen of the display module expands and then returns to its original state when performing layout optimization. When displaying the effect (2030) in which the screen of the display module expands and then returns to its original state, the screen may expand for a predetermined period of time and then return to its original state when layout optimization is completed.

[0217] For example, as shown in FIG. 20d, when performing layout optimization, the electronic device (2000) may display a message notification (2040) in a predetermined area of ​​the display module. The message notification (2040) may be displayed for a predetermined period of time and may disappear as layout optimization is completed.

[0218] For example, in operation 1940, the electronic device may perform layout optimization based on the usage pattern and the display size using the second AI model.

[0219] The electronic device may perform the layout optimization on at least one of a home screen, an app screen, or a lock screen based on the usage pattern and the display size.

[0220] The electronic device can perform the above layout optimization including at least one of: arranging app icons on the home screen, resizing app icons, arranging widgets, changing widget configuration, setting app foldering, changing a wall page image, or editing a status bar.

[0221] In this way, according to the present disclosure, the electronic device and its driving method can efficiently use screen space by optimizing the layout of a home screen, an app screen, or a lock screen as the display size expands or contracts.

[0222] In addition, according to the present disclosure, the electronic device and its driving method can provide a user interface that is adaptive to a situational usage scenario or a user's intention to change the screen size by analyzing various usage environments and learning a user's usage pattern using an AI model.

[0223] In addition, according to the present disclosure, when performing layout optimization, the electronic device and its driving method can notify the user of a change in the user interface according to the layout optimization by displaying a layout optimization execution queue through a display.

[0224] Therefore, according to the present disclosure, the electronic device and its driving method can increase the usability of the sliderable electronic device and maximize user convenience and user satisfaction.

[0225] However, since this has been described above, a duplicate explanation will be omitted.

[0226] The term "module" used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or part of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0227] According to various embodiments, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separated and placed in other components. According to various embodiments, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to various embodiments, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

[0228] According to one aspect of the present disclosure, an electronic device includes a display module including a flexible display having an expandable or contractible display size; a memory storing instructions; and at least one processor operatively connected to the memory and the display module, wherein the instructions, when individually and / or collectively executed by the at least one processor, cause the electronic device to determine a usage pattern of a user of the electronic device according to a usage environment using a first artificial intelligence model, detect a display size of the flexible display based on a change in a state of the display module, and perform layout optimization based on the usage pattern and the display size using a second artificial intelligence model.

[0229] According to one aspect of the present disclosure, the instructions, when individually and / or collectively executed by at least one processor, cause the electronic device to perform the layout optimization on at least one of a home screen, an application screen, or a lock screen, including at least one of: (i) application icon arrangement, (ii) application icon resizing, (iii) widget arrangement, (iv) widget configuration change, (v) application folder setting, (vi) month page image change, or (vi) status bar editing.

[0230] According to one aspect of the present disclosure, the state of the display module includes one of: a first state providing a first display area of ​​the flexible display having a first size; a second state providing a second display area of ​​the flexible display having a second size different from the first size; and an intermediate state providing a third display area of ​​the flexible display having a third size between the first size and the second size.

[0231] According to one aspect of the present disclosure, the instructions, when individually and / or collectively executed by at least one processor, cause the electronic device to analyze the usage environment including at least one of (i) the display size, (ii) location, (iii) time, (iv) movement speed, (v) network status, (vi) battery level, (vii) weather, (viii) ambient light, (ix) ambient noise, or (x) ambient temperature.

[0232] According to one aspect of the present disclosure, the instructions, when individually and / or collectively executed by at least one processor, cause the electronic device to determine the usage pattern including at least one of (i) type of application used, (ii) frequency of application use, (iii) application usage time, (iv) type of widget used, (v) frequency of widget use, (vi) widget usage time, (vii) wallpaper setting, (viii) screen brightness setting, or (ix) resolution setting according to the usage environment.

[0233] According to one aspect of the present disclosure, the instructions, when individually and / or collectively executed by at least one processor, cause the electronic device to perform at least one of: (i) adding an application icon, (ii) enlarging an application icon size, (iii) adding a widget type, (iv) splitting a widget, (v) enlarging a widget size, (vi) unfolding an application, (vii) enlarging a wallpaper image, or (viii) adding a status bar, based on the expanded display size.

[0234] According to one aspect of the present disclosure, the instructions, when individually and / or collectively executed by at least one processor, cause the electronic device to perform at least one of: (i) deleting an application icon, (ii) reducing the size of an application icon, (iii) deleting a widget type, (iv) merging widgets, (v) reducing the size of a widget, (vi) foldering applications, (vii) reducing a wallpaper image, or (viii) deleting a status bar, based on the reduced display size.

[0235] According to one aspect of the present disclosure, the instructions, when individually and / or collectively executed by at least one processor, cause the electronic device to provide additional information that complements the content of at least one of a home screen, an application screen, or a lock screen based on an expansion of the display size, and to provide summary information that compresses the content of at least one of the home screen, the application screen, or the lock screen when the display size is reduced.

[0236] According to one aspect of the present disclosure, the instructions, when individually and / or collectively executed by at least one processor, cause the electronic device to determine a state change intention of the user based on the usage environment and the display size, predict a next action of the user based on the state change intention, and perform the layout optimization based on the predicted next action.

[0237] According to one aspect of the present disclosure, the instructions, when individually and / or collectively executed by at least one processor, cause the electronic device to display, through the display module, at least one layout optimization execution queue that notifies the performance of the layout optimization based on the layout optimization being performed.

[0238] According to one aspect of the present disclosure, a method for driving an electronic device includes: an operation of determining a usage pattern of a user of the electronic device according to a usage environment using a first artificial intelligence model; an operation of detecting a display size of a flexible display of the display module based on a change in a state of the display module, wherein the display size of the flexible display is expandable and contractible; and an operation of performing layout optimization based on the usage pattern and the display size using a second artificial intelligence model.

[0239] According to one aspect of the present disclosure, the operation of performing the layout optimization comprises at least one of (i) arranging application icons, (ii) resizing application icons, (iii) arranging widgets, (iv) changing widget configuration, (v) setting application folders, (vi) changing a month page image, or (vii) editing a status bar on at least one of a home screen, an application screen, or a lock screen.

[0240] According to one aspect of the present disclosure, the state of the display module includes one of: a first state providing a first display area of ​​the flexible display having a first size; a second state providing a second display area of ​​the flexible display having a second size different from the first size; and an intermediate state providing a third display area of ​​the flexible display having a third size between the first size and the second size.

[0241] According to one aspect of the present disclosure, the operation of determining the usage pattern of the user according to the usage environment includes an operation of analyzing the usage environment including at least one of (i) the display size, (ii) location, (iii) time, (iv) movement speed, (v) network status, (vi) remaining battery level, (vii) weather, (viii) ambient illumination, (ix) ambient noise, or (x) ambient temperature.

[0242] According to one aspect of the present disclosure, the operation of determining the usage pattern of the user according to the usage environment includes an operation of determining the usage pattern including at least one of (i) type of application used, (ii) frequency of application use, (iii) application use time, (iv) type of widget used, (v) frequency of widget use, (vi) widget use time, (vii) wallpaper setting, (viii) screen brightness setting, or (ix) resolution setting according to the usage environment.

[0243] According to one aspect of the present disclosure, the operation of performing the layout optimization includes an operation of performing at least one of (i) adding an application icon, (ii) enlarging the application icon size, (iii) adding a widget type, (iv) splitting a widget, (v) enlarging the widget size, (vi) unfolding an application, (vii) enlarging a wallpaper image, or (viii) adding a status bar, based on the expanded display size as the display size is expanded.

[0244] According to one aspect of the present disclosure, the operation of performing the layout optimization includes an operation of performing at least one of (i) deleting an application icon, (ii) reducing the size of an application icon, (iii) deleting a widget type, (iv) merging widgets, (v) reducing the size of a widget, (vi) folderizing applications, (vii) reducing a wallpaper image, or (viii) deleting a status bar, based on the reduced display size.

[0245] According to one aspect of the present disclosure, the operation of performing the layout optimization includes an operation of providing additional information that complements the content of at least one of the home screen, the application screen, or the lock screen based on the display size being expanded, and an operation of providing summary information that compresses the content of at least one of the home screen, the application screen, or the lock screen based on the display size being reduced.

[0246] According to one aspect of the present disclosure, the method further includes an operation of determining a state change intention of the user based on the usage environment and the display size; an operation of predicting a next action of the user based on the state change intention; and an operation of performing layout optimization based on the predicted next action.

[0247] According to one aspect of the present disclosure, the operation of performing the layout optimization includes the operation of displaying at least one layout optimization execution queue that notifies the performance of the layout optimization through the display module.

Claims

1. In an electronic device (101), A display module (160) comprising a flexible display having an expandable or contractible display size; Memory (130) for storing commands; and At least one processor (120) operatively connected to the memory and the display module, The above instructions, when individually and / or collectively executed by at least one processor, cause the electronic device to: Using the first artificial intelligence model, we determine the usage patterns of users of electronic devices according to the usage environment, Detecting the display size of the flexible display based on a change in the state of the display module, Using a second artificial intelligence model, causing layout optimization based on the usage pattern and the display size to be performed, Electronic devices.

2. In paragraph 1, The above instructions, when individually and / or collectively executed by at least one processor, cause the electronic device to: Causing the layout optimization to be performed on at least one of the home screen, application screen, or lock screen, including at least one of application icon placement, application icon resizing, widget placement, widget configuration change, application folder setting, month page image change, or status bar editing. Electronic devices.

3. In paragraph 1 or 2, The above state of the above display module is, A first state providing a first display area of ​​the flexible display having a first size; A second state providing a second display area of ​​the flexible display having a second size different from the first size; and an intermediate state providing a third display area of ​​the flexible display having a third size between the first size and the second size; including one of the following; Electronic devices.

4. In any one of paragraphs 1 to 3, The above instructions, when individually and / or collectively executed by at least one processor, cause the electronic device to: Causing the user to analyze the usage environment including at least one of the display size, location, time, movement speed, network status, battery level, weather, ambient light, ambient noise, or ambient temperature. Electronic devices.

5. In any one of paragraphs 1 to 4, The above instructions, when individually and / or collectively executed by at least one processor, cause the electronic device to: Causing the user to determine the usage pattern including at least one of the type of application used, frequency of application use, application use time, type of widget used, frequency of widget use, widget use time, wallpaper setting, screen brightness setting, or resolution setting according to the usage environment. Electronic devices.

6. In any one of paragraphs 1 to 5, The above instructions, when individually and / or collectively executed by at least one processor, cause the electronic device to: Based on the expanded display size, at least one of adding an application icon, enlarging the application icon size, adding a widget type, splitting a widget, enlarging the widget size, unfolding an application, enlarging a wallpaper image, or adding a status bar is performed based on the expanded display size. Electronic devices.

7. In any one of paragraphs 1 to 6, The above instructions, when individually and / or collectively executed by at least one processor, cause the electronic device to: Based on the reduced display size, causing at least one of application icon deletion, application icon size reduction, widget type deletion, widget consolidation, widget size reduction, application foldering, wallpaper image reduction, or status bar deletion to be performed based on the reduced display size. Electronic devices.

8. In any one of paragraphs 1 to 7, The above instructions, when individually and / or collectively executed by at least one processor, cause the electronic device to: Based on the expansion of the above display size, providing additional information that complements the contents of at least one of the home screen, application screen, or lock screen, When the display size is reduced, causing summary information to be provided that compresses the content of at least one of the home screen, the application screen, or the lock screen. Electronic devices.

9. In any one of paragraphs 1 to 8, The above instructions, when individually and / or collectively executed by at least one processor, cause the electronic device to: Determine the user's intention to change status based on the above usage environment and the above display size, Predict the user's next action based on the state change intention, Causing the layout optimization to be performed based on the predicted next action, Electronic devices.

10. In any one of paragraphs 1 to 9, The above instructions, when individually and / or collectively executed by at least one processor, cause the electronic device to: Based on the above layout optimization being performed, causing at least one layout optimization execution queue to be displayed through the display module, which notifies the performance of the layout optimization. Electronic devices.

11. In a method of driving an electronic device, An operation of determining a user's usage pattern of the electronic device according to the usage environment using a first artificial intelligence model; An operation of detecting a display size of a flexible display of the display module based on a change in the state of the display module, wherein the display size of the flexible display is expandable and contractible; and An operation of performing layout optimization based on the usage pattern and the display size using a second artificial intelligence model; Method of driving an electronic device.

12. In paragraph 11, The action of performing the above layout optimization is: Including at least one of arranging application icons, resizing application icons, arranging widgets, changing widget configuration, setting application folders, changing the month page image, or editing the status bar on at least one of the home screen, application screen, or lock screen. Method of driving an electronic device.

13. In paragraph 11 or 12, The above state of the above display module is, A first state providing a first display area of ​​the flexible display having a first size; A second state providing a second display area of ​​the flexible display having a second size different from the first size; and an intermediate state providing a third display area of ​​the flexible display having a third size between the first size and the second size; including one of the following; Method of driving an electronic device.

14. In any one of paragraphs 11 to 13, The action of determining the usage pattern of the user according to the usage environment is as follows: Comprising an operation of analyzing the usage environment including at least one of the display size, location, time, movement speed, network status, remaining battery level, weather, ambient light, ambient noise, or ambient temperature. Method of driving an electronic device.

15. In any one of paragraphs 11 to 14, The action of determining the usage pattern of the user according to the usage environment is as follows: Including an operation of determining the usage pattern including at least one of the type of application used, frequency of application use, application use time, type of widget used, frequency of widget use, widget use time, wallpaper setting, screen brightness setting, or resolution setting according to the usage environment. Method of driving an electronic device.

Images