Electronic device providing a user interface and method thereof

Abstract

An electronic device according to various embodiments can include a display to display a screen on at least a portion of a display area, a sensor to sense expansion or contraction of the display area, and a processor operatively connected to the display. The processor can be configured to receive a user input related to the screen displayed on at least the portion of the display area, identify expansion of the display area through the sensor, and display an input user interface (UI) including a UI component determined based on the user input on at least a portion of the expanded display area in response to the reception of the user input and the identification of the expansion of the display area.

Description

Technical Field

[0001] Various embodiments relate to an electronic device and a method of controlling the electronic device. Background Technology

[0002] Portable electronic devices are being developed in various forms to ensure an expanded display area without compromising portability. For example, the electronic device can be configured as a sliding electronic device (where the first and second structures unfold relative to each other when used in a position where they overlap), or a foldable electronic device (where the first and second structures match each other to overlap or unfold). Alternatively, the electronic device can be configured in various forms capable of expanding the display area by using a flexible display (e.g., a scrollable display). Summary of the Invention

[0003] Technical issues

[0004] Since the expansion of the display area of ​​an electronic device may change its usability, it is required that the electronic device provide a user interface suitable for the changed usability.

[0005] The technical topics pursued in this disclosure may not be limited to those described above, and other technical topics not mentioned will be clearly understood by those skilled in the art from the following description.

[0006] Solution to the problem

[0007] An electronic device according to various embodiments may include: a display configured to display a screen on at least a portion of a display area; a sensor configured to detect expansion and contraction of the display area; and a processor configured to be operatively connected to the display. The processor may be configured to receive user input related to the screen displayed on at least a portion of the display area, identify expansion of the display area via the sensor, and, in response to receiving the user input and identifying the expansion of the display area, display an input UI including user interface (UI) components determined based on the user input on at least a portion of the expanded display area.

[0008] A method of operating an electronic device including a display according to various embodiments may include: displaying a screen on at least a portion of a display area of ​​the display, receiving user input related to the screen displayed on at least a portion of the display area, identifying an extension of the display area, and, in response to receiving the user input and identifying the extension of the display area, displaying an input UI including UI components determined based on the user input on at least a portion of the extended display area.

[0009] Beneficial effects of the invention

[0010] Electronic devices and their operating methods according to various embodiments can provide a user interface for easily manipulating electronic devices with an extended display area.

[0011] The beneficial effects that can be obtained from this disclosure may not be limited to the effects described above, and other effects not mentioned will be clearly understood by those skilled in the art to which this disclosure pertains through the following description. Attached Figure Description

[0012] Figure 1 This is a front perspective view showing an electronic device in a first state according to various embodiments.

[0013] Figure 2 This is a front perspective view showing an electronic device in a second state according to various embodiments.

[0014] Figure 3 This is a diagram illustrating the flat state of an electronic device according to various other embodiments.

[0015] Figure 4 This is a diagram illustrating the folded state of an electronic device according to various other embodiments.

[0016] Figure 5 This is a block diagram illustrating an electronic device in a network environment according to various embodiments.

[0017] Figure 6 This is a block diagram illustrating the configuration of an electronic device according to various embodiments.

[0018] Figure 7 This is a flowchart illustrating the process of providing a user interface by an electronic device according to various embodiments.

[0019] Figure 8 This is a diagram illustrating examples of screens output by an electronic device according to various embodiments in response to user interaction and expansion of the display area.

[0020] Figure 9 This is a flowchart illustrating the process by which an electronic device, according to various embodiments, provides a user interface based on an object selected by user input.

[0021] Figure 10 This is a diagram illustrating an example of a screen output by an electronic device according to various embodiments when the display area is expanded without user interaction.

[0022] Figure 11 This is a flowchart illustrating the process by which an electronic device, according to various embodiments, provides an input UI for receiving multi-modal input.

[0023] Figure 12This is a diagram illustrating examples of screens displayed by electronic devices to receive complex modal input, according to various embodiments.

[0024] Figure 13 This is a diagram illustrating examples of electronic devices switching the location of the displayed input UI according to various embodiments.

[0025] Figure 14 This is a flowchart illustrating the process by which an electronic device, according to various embodiments, provides an input UI based on the state of an application being executed.

[0026] Figure 15 This is a diagram illustrating an example of a screen displayed by an electronic device according to various embodiments, for providing an input UI based on the state of an application being executed.

[0027] Figure 16 This is a diagram illustrating examples of screens on which electronic devices display altered input UIs according to various embodiments.

[0028] Figure 17 This is a diagram illustrating an example of a screen displayed to provide an input UI based on the layout of an application being executed by an electronic device according to various embodiments.

[0029] Figure 18 This is a diagram illustrating an example of a screen displayed by an electronic device according to various embodiments for providing input UEI in the form of a floating UI.

[0030] Figure 19 This is a flowchart illustrating the process of configuring and displaying an input UI by an electronic device according to various embodiments.

[0031] Figure 20 This is a diagram illustrating an example of a screen displayed by an electronic device according to various embodiments for providing an input UI configured based on user input.

[0032] Figure 21 This is a diagram illustrating an example of a screen displayed by an electronic device according to various embodiments, for providing an input UI when the display area is expanded.

[0033] Figure 22 This is a diagram illustrating an example of a screen displayed by an electronic device according to various embodiments for providing an input UI including a voice input UI component and a handwriting input UI component.

[0034] Figure 23 This is a diagram illustrating an example of a screen displayed by an electronic device according to various embodiments, providing an input UI capable of inputting images.

[0035] Figure 24This is a flowchart illustrating the process by which an electronic device, according to various embodiments, outputs an input UI based on the expansion direction of the display area.

[0036] Figure 25 This is a diagram illustrating an example of a screen displayed by an electronic device according to various embodiments, for providing an input UI based on the extended direction of the display area. Detailed Implementation

[0037] In the following description, various embodiments of the present disclosure will be illustrated with reference to the accompanying drawings. However, this is not intended to limit the present disclosure to the specific embodiments.

[0038] Figure 1 This is a front perspective view showing an electronic device 100 in a first state according to various embodiments. Figure 2 This is a front perspective view showing an electronic device 100 in a second state according to various embodiments.

[0039] According to the various embodiments disclosed in this document, a surface facing a direction substantially the same as the direction facing at least a portion (e.g., the first portion 121) of the flexible display 120 located outside the electronic device 100 may be defined as the front surface of the electronic device 100, and a surface facing the front surface may be defined as the rear surface of the electronic device 100. A surface surrounding the space between the front and rear surfaces may be defined as the side surface of the electronic device 100.

[0040] According to an embodiment, the flexible display 120 may be disposed on at least a portion of the electronic device 100. In an embodiment, the flexible display 120 may be configured to include at least a partially flat shape and at least a partially curved shape. In an embodiment, the flexible display 120 and a slidable housing 110 surrounding at least a portion of the periphery of the flexible display 120 may be disposed on the front surface of the electronic device 100.

[0041] In an embodiment, the slidable housing 110 may form the front surface of the electronic device 100 (e.g., the facing surface of the electronic device 100). Figure 1 and Figure 2 The surface in the +z direction of the middle part), the rear surface (e.g., the facing part of the electronic device 100) Figure 1 and Figure 2 The surface in the -z direction and a portion of the side surface (e.g., the surface connecting the front and rear surfaces of the electronic device 100). In another embodiment, the slidable housing 110 may form a portion of the side and rear surfaces of the electronic device 100.

[0042] In an embodiment, the slidable housing 110 may include a first housing 111 and a second housing 112 movably coupled to the first housing 111 within a predetermined range.

[0043] In one embodiment, the flexible display 120 may include a first portion 121 that can be coupled to a second housing 112, and a second portion 122 that extends from the first portion 121 and is inserted into the electronic device 100.

[0044] In an embodiment, the electronic device 100 may include a first state 100a and a second state 100b. For example, the first state 100a and the second state 100b of the electronic device 100 may be determined based on the relative position of the second housing 112 relative to the sliding housing 110, and the electronic device 100 may be configured to be changeable between the first state and the second state by user manipulation or mechanical operation.

[0045] In various embodiments, the first state 100a of the electronic device 100 may refer to the state before the slidable housing 110 is extended. The second state 100b of the electronic device 100 may refer to the state in which the slidable housing 110 is extended.

[0046] In one embodiment, when the electronic device 100 switches from a first state 100a to a second state 100b according to the movement of the second housing 112, the second portion 122 of the flexible display 120 can be retracted (or exposed) from the interior of the electronic device 100 to the exterior. In various embodiments, the fact that the flexible display 120 is retracted (or exposed) can mean that the flexible display 120 is visible from the exterior of the electronic device 100. In another embodiment, when the electronic device 100 switches from a second state 100b to a first state 100a according to the movement of the second housing 112, the second portion 122 of the flexible display 120 can be inserted into the electronic device 100. In various embodiments, the fact that the flexible display 120 is inserted can mean that the flexible display 120 is not visible from the exterior of the electronic device.

[0047] refer to Figure 3 and Figure 4 In an embodiment, the electronic device 30 may include a foldable housing 400, a hinge cover 430 covering a foldable portion of the foldable housing, and a flexible or foldable display 300 (hereinafter referred to as "display" 100) disposed within the space formed by the foldable housing 400. In this document, the surface on which the display 100 is disposed is defined as a first surface or front surface of the electronic device 30. The surface opposite the front surface is defined as a second surface or rear surface of the electronic device 30. Additionally, the surface surrounding the space between the front and rear surfaces is defined as a third surface or side surface of the electronic device 30.

[0048] In an embodiment, the foldable housing 400 may include a first housing structure 410, a second housing structure 420 including a sensor region 424, a first rear cover 480, and a second rear cover 590. The foldable housing 400 of the electronic device 30 is not limited to... Figure 3 and Figure 4 The shapes and combinations shown can be achieved through combinations and / or couplings of other shapes or components. For example, in another embodiment, the first housing structure 410 and the first rear cover 480 can be integrally formed, and the second housing structure 420 and the second rear cover 490 can be integrally formed.

[0049] In the illustrated embodiment, the first housing structure 410 and the second housing structure 420 may be arranged on opposite sides of the folding axis (axis A) and may have a shape that is substantially symmetrical to each other with respect to the folding axis A. As will be described later, the angle or distance between the first housing structure 410 and the second housing structure 420 may vary depending on whether the electronic device is in an extended state, a folded state, or an intermediate state. In the illustrated embodiment, unlike the first housing structure 410, the second housing structure 420 may additionally include a sensor region 424 in which various sensors are arranged; however, the second housing structure 420 and the first housing structure 410 may have mutually symmetrical shapes in other regions.

[0050] In an embodiment, such as Figure 3 As shown, the first housing structure 410 and the second housing structure 420 can together form a recess for accommodating the display 300. In the illustrated embodiment, due to the sensor region 424, the recess can have two or more different widths in the direction perpendicular to the folding axis A.

[0051] For example, the recess may have (1) a first width w1 between a first portion 410a of the first housing structure 410 parallel to the folding axis A and a first portion 420a forming around the sensor region 424 of the second housing structure 420; and (2) a second width w2 formed by the second portion 410b of the first housing structure 410 and the second portion 420b of the second housing structure 420, which does not correspond to the sensor region 424 and is parallel to the folding axis A. In this case, the second width w2 may be formed to be longer than the first width w1. In other words, the first portion 410a of the first housing structure 410 and the first portion 420a of the second housing structure 420, having mutually asymmetrical shapes, may form the first width w1 of the recess, and the second portion 410b of the first housing structure 410 and the second portion 420b of the second housing structure 420, having mutually symmetrical shapes, may form the second width w2 of the recess. In embodiments, the first portion 420a and the second portion 420b of the second housing structure 420 may have different distances from the folding axis A. The width of the recess is not limited to the example shown. In various embodiments, the recess may have multiple widths due to the shape of the sensor region 424 or the asymmetrical shape of the first housing structure 410 and the second housing structure 420.

[0052] In an embodiment, at least a portion of the first housing structure 410 and the second housing structure 420 may be made of a metallic or non-metallic material whose rigidity is selected to support the display 100.

[0053] In one embodiment, the sensor region 424 may be formed as a predetermined region having a corner adjacent to the second housing structure 420. However, the arrangement, shape, and size of the sensor region 424 are not limited to the example shown. For example, in another embodiment, the sensor region 424 may be provided at another corner of the second housing structure 420 or in any region between the top and bottom corners of the second housing structure 420. In one embodiment, components built into the electronic device 30 for performing various functions may be exposed on the front or rear surface of the electronic device 30 through the sensor region 424 or through one or more openings provided in the sensor region 424. In various embodiments, the components may include various types of sensors. The sensors may include at least one of, for example, a front-facing camera, a receiver, and a proximity sensor.

[0054] The first rear cover 480 may be disposed on one side of the folding axis on the rear surface of the electronic device 30, and may have a generally rectangular periphery, which may be enclosed by the first housing structure 410. Similarly, the second rear cover 490 may be disposed on the other side of the folding axis on the rear surface of the electronic device 30, and the periphery of the second rear cover 490 may be enclosed by the second housing structure 420.

[0055] In the illustrated embodiment, the first rear cover 480 and the second rear cover 490 may have a shape that is substantially symmetrical with respect to the folding axis (axis A). However, the first rear cover 480 and the second rear cover 490 do not necessarily have symmetrical shapes, and in another embodiment, the electronic device 30 may include a first rear cover 480 and a second rear cover 490 having various shapes. In another embodiment, the first rear cover 490 may be integrally formed with the first housing structure 410, and the second rear cover 490 may be integrally formed with the second housing structure 420.

[0056] In embodiments, the first rear cover 480, the second rear cover 490, the first housing structure 410, and the second housing structure 420 may form spaces in which various components of the electronic device 30 (e.g., printed circuit boards or batteries) are arranged. In embodiments, one or more components may be arranged or visually exposed on the rear surface of the electronic device 30. For example, at least a portion of the sub-display 390 may be visually exposed through a first rear region 482 of the first rear cover 480. In another embodiment, one or more components or sensors may be visually exposed through a second rear region 492 of the second rear cover 490. In various embodiments, the sensors may include a proximity sensor and / or a rear camera.

[0057] refer to Figure 4 The hinge cover 430 may be disposed between the first housing structure 410 and the second housing structure 420 to cover internal components (e.g., the hinge structure). In embodiments, depending on the state of the electronic device 30 (flat or folded), the hinge cover 430 may be covered by a portion of the first housing structure 410 and the second housing structure 420, or it may be exposed to the outside.

[0058] For example, such as Figure 3 As shown, when the electronic device 30 is in a flat state, the hinge cover 430 may not be exposed because it is covered by the first housing structure 410 and the second housing structure 420. For example, as Figure 4 As shown, when the electronic device 30 is in a folded state (e.g., a fully folded state), the hinge cover 430 may be exposed outside the space between the first housing structure 410 and the second housing structure 420. For example, when the first housing structure 410 and the second housing structure 420 are in an intermediate state in which they are folded at an angle, the hinge cover 430 may be partially exposed outside the space between the first housing structure 410 and the second housing structure 420. In this case, the exposed area may be narrower than the area exposed in the fully folded state. In an embodiment, the hinge cover 430 may be formed to include a curved surface.

[0059] The display 300 can be disposed in the space formed by the foldable housing 400. For example, the display 300 can be located in a recess formed by the foldable housing 400 and can constitute most of the front surface of the electronic device 30.

[0060] The front surface of the electronic device 30 may include the display 300, a portion of the first housing structure 410 adjacent to the display 300, and a portion of the second housing structure 420. The rear surface of the electronic device 30 may include a first rear cover 480, a portion of the first housing structure 410 adjacent to the first rear cover 480, a second rear cover 490, and a portion of the second housing structure 420 adjacent to the second rear cover 490.

[0061] Display 300 may be configured as a display whose at least a portion of a surface can be converted to a flat or curved surface. In an embodiment, display 300 may include a folded region 303, and a side disposed relative to the folded region 303 ( Figure 3 The first region 301 on the left side of the folded region 303, and the region on the other side thereof ( Figure 3 The second region 302 is located to the right of the folded region 303.

[0062] Figure 3 The area division of the display 300 shown is exemplary, and depending on the structure or function of the display, the display 300 may be divided into multiple (e.g., four or more or two) areas. For example, in Figure 3 In the embodiment shown, the area of ​​display 300 may be divided by a folded region 303 extending parallel to the y-axis or the folding axis (A-axis), but in another embodiment, the area of ​​display 300 may be divided relative to another folded region (e.g., a folded region parallel to the x-axis) or another folding axis (e.g., a folding axis parallel to the x-axis).

[0063] The first region 301 and the second region 302 may have generally symmetrical shapes with respect to the folded region 303. However, unlike the first region 301, the second region 302 may include cuts that form a secure sensor region 424. The first region 301 and the second region 302 may include portions that are symmetrical to each other and portions that are asymmetrical to each other.

[0064] The operation of the first housing structure 410 and the second housing structure 420 according to the state of the electronic device 30 (e.g., flat state and folded state) and each area of ​​the display 300 will be described.

[0065] In an embodiment, when the electronic device 30 is in a flat state (e.g., Figure 3The first housing structure 410 and the second housing structure 410 can be arranged to face the same direction while forming a 180-degree angle between them. The surfaces of the first region 301 and the second region 302 of the display 300 can form a 180-degree angle between them and can face the same direction (e.g., the direction of the front surface of the electronic device). The folding region 303 can form the same plane as the first region 301 and the second region 302.

[0066] In an embodiment, when the electronic device 30 is in a folded state (e.g., Figure 4 The first housing structure 410 and the second housing structure 420 can face each other. The surfaces of the first region 301 and the second region 302 of the display 300 can form narrow angles (e.g., between 0 and 10 degrees) and can face each other. At least a portion of the folded region 103 can form a curved surface with a predetermined curvature.

[0067] In an embodiment, when the electronic device 30 is in an intermediate state (e.g., in...), Figure 3 flat state and Figure 4 Between the folded states, the first housing structure 410 and the second housing structure 420 can be arranged to form an angle between them. The surfaces of the first region 301 and the second region 302 of the display 300 can form an angle greater than that in the folded state and less than that in the flat state. At least a portion of the folded region 103 can be formed by a curved surface with a predetermined curvature, and the curvature can be less than that in the folded state.

[0068] Electronic device 30 may include sensors for detecting the state of electronic device 30 (e.g., folded state, intermediate state, or flat state). For example, electronic device 30 may include sensors for detecting the angle between the first housing structure 410 and the second housing structure 420 at the hinge portion. As another example, electronic device 30 may include sensors for determining whether electronic device 30 is in a folded state by using a magnetic material disposed on the first housing structure 410 and a Hall sensor disposed on the second housing structure 420.

[0069] The electronic device 30 can determine a display area, i.e., the area where the display screen is located. According to an embodiment, the electronic device 30 can determine the display area based on the state of the electronic device (e.g., a folded state, an intermediate state, or a flat state). For example, when the electronic device 30 is in a folded state, the area where the sub-display 190 can output the screen can be determined as the display area, and when the electronic device 30 is in a flat state, the display area can be expanded such that the area where the display 300 outputs the screen is the display area. As another example, when the electronic device 30 is in an intermediate state, the display area can be determined as an area having an angle determined based on the angle between the first housing structure 410 and the second housing structure 420, and when the electronic device 30 is in a flat state, the display area can be expanded to the entire display 300.

[0070] Figures 1 to 4 The shapes of the electronic devices shown are used to illustrate examples of electronic devices capable of expanding the display area, and the shapes of the electronic devices are not limited to those shown. Figures 1 to 4 The shape shown. For example, an electronic device may include an electronic device with a scrollable display. An electronic device with a scrollable display can define the remainder of the display, excluding its scrollable portion, as the display area.

[0071] Figure 5 This is a block diagram illustrating an electronic device 501 in a network environment 500 according to various embodiments. Reference Figure 5 In network environment 500, electronic device 501 can communicate with electronic device 502 via a first network 598 (e.g., a short-range wireless communication network), or with at least one of electronic device 504 or server 508 via a second network 599 (e.g., a long-range wireless communication network). According to an embodiment, electronic device 501 can communicate with electronic device 504 via server 508. According to an embodiment, electronic device 501 may include a processor 520, memory 530, input module 550, sound output module 555, display module 560, audio module 570, sensor module 576, interface 577, connection terminal 578, haptic module 579, camera module 580, power management module 588, battery 589, communication module 590, user identification module (SIM) 596, or antenna module 597. In some embodiments, at least one of the above components (e.g., connection terminal 578) may be omitted from electronic device 501, or one or more other components may be added to electronic device 501. In some embodiments, some of the components described above (e.g., sensor module 576, camera module 580, or antenna module 597) may be implemented as a single integrated component (e.g., display module 560).

[0072] Processor 520 may run software (e.g., program 540) to control at least one other component (e.g., hardware or software component) of electronic device 501 connected to processor 520, and may perform various data processing or calculations. According to one embodiment, as at least part of the data processing or calculation, processor 520 may store commands or data received from another component (e.g., sensor module 576 or communication module 590) in volatile memory 532, process the commands or data stored in volatile memory 532, and store the resulting data in non-volatile memory 534. According to embodiments, processor 520 may include a main processor 521 (e.g., central processing unit (CPU) or application processor (AP)) or an auxiliary processor 523 (e.g., graphics processing unit (GPU), neural processing unit (NPU), image signal processor (ISP), sensor central processor, or communication processor (CP)) that is operationally independent of or combined with the main processor 521. For example, when electronic device 501 includes a main processor 521 and an auxiliary processor 523, the auxiliary processor 523 may be adapted to consume less power than the main processor 521, or to be adapted to perform a specific function. The auxiliary processor 523 may be implemented separately from the main processor 521, or may be implemented as part of the main processor 521.

[0073] When the main processor 521 is inactive (e.g., in sleep mode), the auxiliary processor 523 (rather than the main processor 521) can control at least some of the functions or states associated with at least one component of the electronic device 501 (e.g., display module 560, sensor module 576, or communication module 590), or when the main processor 521 is active (e.g., running an application), the auxiliary processor 523 can work with the main processor 521 to control at least some of the functions or states associated with at least one component of the electronic device 501 (e.g., display module 560, sensor module 576, or communication module 590). According to embodiments, the auxiliary processor 523 (e.g., an image signal processor or a communication processor) may be implemented as part of another component (e.g., camera module 580 or communication module 590) functionally associated with the auxiliary processor 523. According to embodiments, the auxiliary processor 523 (e.g., a neural processing unit) may include hardware architecture dedicated to artificial intelligence model processing. Artificial intelligence models can be generated through machine learning. For example, such learning can be performed via electronic device 501 where artificial intelligence is performed or via a separate server (e.g., server 508). The learning algorithm may include, but is not limited to, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning. The artificial intelligence model may include multiple layers of artificial neural networks. The artificial neural network may be a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), or a deep Q-network, or a combination of two or more thereof, but is not limited thereto. Additionally or optionally, the artificial intelligence model may include software structures in addition to hardware structures.

[0074] The memory 530 may store various data used by at least one component of the electronic device 501 (e.g., processor 520 or sensor module 576). The various data may include, for example, software (e.g., program 540) and input or output data for commands associated with it. The memory 530 may include volatile memory 532 or non-volatile memory 534.

[0075] The program 540 may be stored as software in the memory 530, and the program 540 may include, for example, an operating system (OS) 542, middleware 544, or application 546.

[0076] The input module 550 can receive commands or data from outside the electronic device 501 (e.g., a user) that will be used by other components of the electronic device 501 (e.g., processor 520). The input module 550 may include, for example, a microphone, mouse, keyboard, keys (e.g., buttons), or digital pen (e.g., stylus).

[0077] The sound output module 555 can output sound signals to the outside of the electronic device 501. The sound output module 555 may include, for example, a speaker or a receiver. The speaker can be used for general purposes such as playing multimedia or playing records. The receiver can be used to receive incoming calls. According to embodiments, the receiver can be implemented separately from the speaker or as part of the speaker.

[0078] Display module 560 can visually provide information to the outside of electronic device 501 (e.g., to a user). Display device 560 may include, for example, a display, a holographic device, or a projector, and control circuitry for controlling a respective one of the display, holographic device, and projector. According to an embodiment, display module 560 may include a touch sensor adapted to detect touch or a pressure sensor adapted to measure the intensity of the force caused by touch.

[0079] The audio module 570 can convert sound into electrical signals and vice versa. According to an embodiment, the audio module 570 can obtain sound via the input module 550, or output sound via the sound output module 555 or headphones of an external electronic device (e.g., electronic device 502) that is directly (e.g., wired) or wirelessly connected to the electronic device 501.

[0080] Sensor module 576 can detect the operating state of electronic device 501 (e.g., power or temperature) or the environmental state outside electronic device 501 (e.g., user state), and then generate an electrical signal or data value corresponding to the detected state. According to embodiments, sensor module 576 may include, for example, a gesture sensor, gyroscope sensor, atmospheric pressure sensor, magnetic sensor, accelerometer, grip sensor, proximity sensor, color sensor, infrared (IR) sensor, biometric sensor, temperature sensor, humidity sensor, or illuminance sensor.

[0081] Interface 577 may support one or more specific protocols used to enable electronic device 501 to connect directly (e.g., wired) or wirelessly to external electronic devices (e.g., electronic device 502). According to embodiments, interface 577 may include, for example, a High Definition Multimedia Interface (HDMI), a Universal Serial Bus (USB) interface, a Secure Digital Card (SD) interface, or an audio interface.

[0082] Connection end 578 may include a connector, through which electronic device 501 can be physically connected to an external electronic device (e.g., electronic device 502). According to embodiments, connection end 578 may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0083] The haptic module 579 can convert electrical signals into mechanical stimulation (e.g., vibration or motion) or electrical stimulation that can be recognized by a user through his touch or kinesthesia. According to embodiments, the haptic module 579 may include, for example, a motor, a piezoelectric element, or an electrical stimulator.

[0084] Camera module 580 can capture still or moving images. According to an embodiment, camera module 580 may include one or more lenses, an image sensor, an image signal processor, or a flash.

[0085] The power management module 588 manages the power supply to the electronic device 501. According to an embodiment, the power management module 588 may be implemented as at least part of, for example, a power management integrated circuit (PMIC).

[0086] Battery 589 can power at least one component of electronic device 501. According to an embodiment, battery 589 may include, for example, a non-rechargeable primary battery, a rechargeable rechargeable battery, or a fuel cell.

[0087] Communication module 590 can support the establishment of a direct (e.g., wired) or wireless communication channel between electronic device 501 and external electronic devices (e.g., electronic device 502, electronic device 504, or server 508), and perform communication via the established communication channel. Communication module 590 may include one or more communication processors capable of operating independently of processor 520 (e.g., application processor (AP)) and support direct (e.g., wired) or wireless communication. According to embodiments, communication module 590 may include wireless communication module 592 (e.g., cellular communication module, short-range wireless communication module, or Global Navigation Satellite System (GNSS) communication module) or wired communication module 594 (e.g., local area network (LAN) communication module or power line communication (PLC) module). One of these communication modules can communicate with an external electronic device via a first network 598 (e.g., a short-range communication network such as Bluetooth, Wi-Fi Direct, or Infrared Data Association (IrDA)) or a second network 599 (e.g., a long-range communication network such as a traditional cellular network, 5G network, next-generation communication network, the Internet, or a computer network (e.g., a LAN or a wide area network (WAN))). These various types of communication modules can be implemented as a single component (e.g., a single chip) or as multiple components separate from each other (e.g., multiple chips). The wireless communication module 592 can identify and verify the electronic device 501 in the communication network (such as the first network 598 or the second network 599) using user information (e.g., the International Mobile Subscriber Identity (IMSI)) stored in the user identification module 596.

[0088] Wireless communication module 592 can support 5G networks following 4G networks and next-generation communication technologies (such as new radio (NR) access technologies). NR access technologies can support enhanced mobile broadband (eMBB), massive machine-type communication (mMTC), or ultra-reliable low-latency communication (URLLC). Wireless communication module 592 can support high-frequency bands (e.g., millimeter-wave bands) to achieve, for example, high data transmission rates. Wireless communication module 592 can support various technologies used to ensure performance in high-frequency bands, such as, for example, beamforming, massive MIMO, full-dimensional MIMO (FD-MIMO), array antennas, analog beamforming, or massive antennas. Wireless communication module 592 can support various requirements specified in electronic device 501, external electronic device (e.g., electronic device 504), or network system (e.g., second network 599). According to an embodiment, the wireless communication module 592 may support peak data rates (e.g., 20 Gbps or greater) for implementing eMBB, lost coverage (e.g., 164 dB or less) for implementing mMTC, or U-plane latency (e.g., 0.5 ms or less for each of the downlink (DL) and uplink (UL), or 1 ms or less round trip) for implementing URLLC.

[0089] Antenna module 597 can transmit or receive signals or power to or from the exterior of electronic device 501 (e.g., external electronic device). According to an embodiment, antenna module 597 may include an antenna comprising a radiating element formed of a conductive material or conductive pattern formed in or on a substrate (e.g., a printed circuit board (PCB)). According to an embodiment, antenna module 597 may include multiple antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication scheme used in a communication network (such as a first network 598 or a second network 599) can be selected from the multiple antennas by, for example, communication module 590 (e.g., wireless communication module 592). Signals or power can then be transmitted or received between communication module 590 and external electronic device via the selected at least one antenna. According to an embodiment, additional components besides the radiating element (e.g., a radio frequency integrated circuit (RFIC)) may be additionally incorporated into antenna module 597.

[0090] According to various embodiments, antenna module 597 can form a millimeter-wave antenna module. According to embodiments, the millimeter-wave antenna module may include a printed circuit board, a radio frequency integrated circuit (RFIC), and multiple antennas (e.g., an array antenna), wherein the RFIC is disposed on or adjacent to a first surface (e.g., a bottom surface) of the printed circuit board and is capable of supporting a specified high-frequency band (e.g., a millimeter-wave band), and the multiple antennas are disposed on or adjacent to a second surface (e.g., a top or side surface) of the printed circuit board and are capable of transmitting or receiving signals in the specified high-frequency band.

[0091] At least some of the aforementioned components can be interconnected and communicate signals (e.g., commands or data) between them via an inter-peripheral communication scheme (e.g., bus, general purpose input / output (GPIO), serial peripheral interface (SPI), or mobile industrial processor interface (MIPI)).

[0092] According to an embodiment, commands or data can be sent or received between electronic device 501 and external electronic device 504 via server 508 connected to a second network 599. Each of electronic device 502 or electronic device 504 can be a device of the same type as electronic device 501, or a device of a different type. According to an embodiment, all or some operations that would be performed on electronic device 501 can be performed on one or more of external electronic devices 502, external electronic devices 504, or server 508. For example, if electronic device 501 is required to automatically perform a function or service, or is required to perform a function or service in response to a request from a user or another device, electronic device 501 may request the one or more external electronic devices to perform at least a portion of the function or service, instead of running the function or service, or electronic device 501 may request the one or more external electronic devices to perform at least a portion of the function or service in addition to running the function or service. Upon receiving the request, one or more external electronic devices may perform at least a portion of the requested function or service, or perform additional functions or services related to the request, and transmit the result of the execution to electronic device 501. Electronic device 501 may provide the result as at least a partial response to the request, with or without further processing. For this purpose, technologies such as cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing may be used. Electronic device 501 may use, for example, distributed computing or mobile edge computing to provide ultra-low latency services. In another embodiment, external electronic device 504 may include an Internet of Things (IoT) device. Server 508 may be an intelligent server using machine learning and / or neural networks. According to embodiments, external electronic device 504 or server 508 may be included in a second network 599. Electronic device 501 may be applied to intelligent services based on 5G communication technology or IoT-related technologies (e.g., smart homes, smart cities, smart cars, or healthcare).

[0093] The electronic device according to various embodiments can be one of a variety of types of electronic devices. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. According to embodiments of this disclosure, the electronic device is not limited to those described above.

[0094] It should be understood that the various embodiments of this disclosure and the terminology used therein are not intended to limit the technical features set forth herein to the specific embodiments, but rather to include various changes, equivalents, or substitutions to the respective embodiments. In the description of the drawings, similar reference numerals may be used to refer to similar or related elements. It will be understood that nouns in the singular form corresponding to terms may include one or more things unless the relevant context clearly indicates otherwise. As used herein, each of the phrases such as “A or B,” “at least one of A and B,” “at least one of A or B,” “A, B, or C,” “at least one of A, B, and C,” and “at least one of A, B, or C” may include any one or all possible combinations of the items enumerated together with the corresponding phrase among the plurality of phrases. As used herein, terms such as “first” and “second” or “first” and “second” may be used to simply distinguish the respective component from another component and do not limit the component in other respects (e.g., importance or order). It will be understood that, whether the terms “operably” or “communically” are used or not, if an element (e.g., a first element) is referred to as “combined with another element (e.g., a second element),” “combined to another element (e.g., a second element),” “connected to another element (e.g., a second element),” or “attached to another element (e.g., a second element)”, it means that the first element can be directly (e.g., wiredly) connected to the second element, wirelessly connected to the second element, or connected to the second element via a third element.

[0095] As used in connection with various embodiments of this disclosure, the term "module" may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with other terms (e.g., "logic," "logic block," "part," or "circuit"). A module may be a single integrated component adapted to perform one or more functions, or the smallest unit or part of such a single integrated component. For example, according to embodiments, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0096] The various embodiments set forth herein can be implemented as software (e.g., program 540) comprising one or more instructions readable by a machine (e.g., electronic device 501) stored in a storage medium (e.g., internal memory 536 or external memory 538). For example, under the control of a processor, the processor (e.g., processor 520) of the machine (e.g., electronic device 501) can invoke and execute at least one of the one or more instructions stored in the storage medium, with or without the use of one or more other components. This enables the machine to operate to perform at least one function according to the invoked at least one instruction. The one or more instructions may include code generated by a compiler or code executable by an interpreter. Machine-readable storage media may be provided in the form of non-transitory storage media. The term "non-transitory" means only that the storage medium is a tangible device and does not include signals (e.g., electromagnetic waves), but this term does not distinguish between data being stored semi-permanently in the storage medium and data being temporarily stored in the storage medium.

[0097] According to embodiments, methods according to various embodiments of this disclosure may be included and provided in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., a compact disk read-only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) via an app store (e.g., the Play Store™), or may be distributed directly between two user devices (e.g., smartphones) (e.g., downloaded or uploaded). If distributed online, at least a portion of the computer program product may be temporarily generated, or at least a portion of the computer program product may be stored at least temporarily in a machine-readable storage medium (such as the memory of a manufacturer's server, an app store's server, or a forwarding server).

[0098] According to various embodiments, each of the above-described components (e.g., a module or program) may include a single entity or multiple entities, and some of the multiple entities may be separately disposed in different components. According to various embodiments, one or more of the above-described components may be omitted, or one or more other components may be added. Optionally or additionally, multiple components (e.g., modules or programs) may be integrated into a single component. In this case, according to various embodiments, the integrated component may still perform the one or more functions of each of the multiple components in the same or similar manner as the corresponding component of the multiple components performed one or more functions before integration. According to various embodiments, the operations performed by a module, program, or other component may be performed sequentially, in parallel, repeatedly, or heuristically, or one or more of the operations may be run in a different order or omitted, or one or more other operations may be added.

[0099] Figure 6 This illustrates an electronic device 101 according to various embodiments (e.g., Figure 1 and Figure 2 Electronic devices 100 Figure 3 and Figure 4 Electronic devices 30 or Figure 5 A block diagram of the configuration of the electronic device 101.

[0100] Electronic device 101 may include display 610 (e.g., Figure 1 and 2 Flexible display 120 Figure 3 and 4 300 or Figure 5 The display module 560), and the processor 620 (e.g., Figure 5 The processor 520 and the sensor 630 (e.g., sensor module 576). Figure 6 Embodiments for explaining the components of electronic device 101, and the components included in electronic device 101 are not limited to Figure 6 The components shown. For example. Figure 6 Components of the electronic device 101 shown may be replaced with other components, or additional components may be included in the electronic device 101. For example, the electronic device 101 according to various embodiments may further include a camera 640 (e.g., Figure 5 (Camera module 580). For example, the processor 220 of the electronic device 101 according to various embodiments may be replaced, such that data can be processed by an external device outside the electronic device 101 using cloud computing technology, and the components of the electronic device 101 can be controlled.

[0101] According to various embodiments, processor 620 may execute operations loaded into memory (e.g., Figure 5 The processor 620 executes instructions contained in the memory 530 to control the operation of components of the electronic device 101 (e.g., display 610). The processor 620 can execute instructions contained in software to control at least one other component operatively connected to the processor 620. The processor 620 can fetch instructions and interpret the fetched instructions to process data or perform calculations. It is understood that the operation of the electronic device 101 mentioned herein is performed by the processor 220 executing the instructions.

[0102] Display 610 can visually display (or provide or output) images. For example, processor 620 of electronic device 101 can display an application execution screen or a keypad screen via display 610. When display 610 is a device with a touch sensor capable of detecting the touch of an object, processor 620 can receive touch input to the screen displayed on the display from the touch sensor of display 610. Touch input may include, for example, coordinate values ​​indicating the type of touch event and the location where the touch input is received. However, the type of touch input is not limited to this.

[0103] The electronic device 101 according to various embodiments can expand or shrink the display area of ​​the display 610. The display area can refer to the area where the screen is displayed on the display 610. For example, in a first state, only the first portion 121 of the display 120 can be exposed to the outside, and Figure 1 The processor of the illustrated electronic device 100 can determine the first region 121 as the display area of ​​the display screen therein. Figure 2 In the second state shown, the processor of the electronic device 100 can determine the area obtained by combining the first region 121 and the second portion 122 inserted into the housing as the display area. For example, in... Figure 3 and Figure 4 The electronic device 30 shown utilizes a sub-display 390 as... Figure 4 The display area shown in the diagram displays the state of the screen when the state of the electronic device 30 is switched. Figure 3 In the flat state shown, the display 300 can be used as the display area of ​​the display screen.

[0104] Sensor 630 may include a sensor for detecting an extension of the display area. In this document, detecting an extension of the display area can mean detecting an operation that extends the display area. For example, sensor 630 may be configured to detect... Figure 1 and Figure 2 The illustrated electronic device 100 performs mechanical operation and detects deformation between a first state and a second state. For example, sensor 630 may include a Hall sensor for detecting deformation from... Figure 4 The folded state to Figure 3 The transition from a flat state. As another example, electronic device 101 may include a hardware button (not shown) that allows mechanical operation. When the user presses the hardware button, it can be activated from... Figure 1 The first state to Figure 2 The second state or from Figure 4 The folded state to Figure 3 The flattened state undergoes mechanical operation. Sensor 630 may be a component for detecting that a hardware button has been pressed. As another example, electronic device 101 may display a software button via display 610 to allow for expansion of the display. When user input (e.g., touch input or mouse click) for selecting a software button is received, processor 620 may determine that the display area has been expanded. That is, electronic device 630 according to various embodiments may be configured to detect expansion of the display area without including sensor 630 as a separate hardware configuration.

[0105] Processor 620 may display a screen on at least a portion of the display area of ​​display 610. For example, processor 620 may execute a messaging application installed in electronic device 101 and display the execution screen of the messaging application on at least a portion of the display area. Processor 620 may receive user input to the display screen. For example, when display 610 includes a touchscreen with a touch sensor, electronic device may receive touch input to a text input field included in the execution screen of the messaging application. The text input field may refer to an area in which text input by the user is displayed. According to embodiments, user input may be used in various forms. For example, user input may include tap input (i.e., touch input with a short hold period), long press input with a hold period longer than a specified period, swipe input where the touch input passes through the position of an object displayed on the screen while the touch input is held, drag-and-drop input where the touch position moves from the position of the object to the endpoint, or for recognition via a microphone (e.g., Figure 5 The input module 550 receives the user's voice input.

[0106] In response to the recognition of an expansion of the display area via sensor 630, processor 620, according to various embodiments, may determine user interface (UI) components based on user input received prior to the expansion of the display area. For example, when the user input is for selecting a text input field, processor 620 may select a UI component including a keypad. The UI component may be referred to as a component constituting the input UI. For example, the UI component may include at least one of the following: a virtual keypad area for receiving text input, a voice input status display area associated with receiving voice input, a handwriting area for receiving handwritten input from the user, an image list area for selecting an image stored in an electronic device, or a preview area for inputting an image captured by camera 640. Processor 620 may display the input UI including the determined UI components on at least a portion of the expanded display area of ​​display 610.

[0107] Figure 7 This illustrates electronic devices according to various embodiments (e.g., Figure 1 and Figure 2 Electronic devices 100 Figure 3 and Figure 4 30 electronic devices Figure 5 Electronic device 101 or Figure 6 The flowchart 700 shows the process of providing a UI for the electronic device 101.

[0108] The electronic device can perform operation 710 of displaying a screen on at least a portion of the display area of ​​a display. According to various embodiments, the screen displayed in operation 710 may include objects related to the functions provided by the electronic device. For example, the electronic device may display a screen including at least one of the following: an input field for displaying information entered by user input, a keypad for inputting text, an emoji list for inputting emojis, or a photo input icon for inputting a photo.

[0109] The electronic device can perform operation 720, which receives user input. For example, the user input for operation 720 could be input for selecting an object included in the screen displayed in operation 710. The user input for operation 720 can be generated based on another interaction between the electronic device and the user. The electronic device can perform operation 730, which identifies an extension of the display area. For example, when the state of the electronic device changes from... Figure 1 The first state transition to Figure 2 In the second state, the electronic device can identify whether the area of ​​the screen displayed on the monitor has expanded.

[0110] In response to the recognition of an expansion of the display area, the electronic device can perform operation 740 of displaying an input UI. Operation 740 may include determining UI components based on user input and configuring an input UI including the determined UI components. According to an embodiment, the UI components included in the input UI may be an expansion of an object selected by user input. For example, when the display area is expanded after receiving user input for selecting a text input area, the electronic device may display an input UI on the display including a keypad for inputting text and an expanded text input area.

[0111] Figure 8 This illustrates electronic devices according to various embodiments (e.g., Figure 1 and Figure 2 Electronic devices 100 Figure 3 and Figure 4 30 electronic devices Figure 5 Electronic device 101 or Figure 6 A diagram showing an example of a screen output by an electronic device 101 in response to user interaction or expansion of the display area.

[0112] The electronic device can display a screen 810 including a text input area 811. The electronic device can recognize the expansion of the display area after receiving touch input 812 for selecting the text input area 811 or at the same time as receiving touch input.

[0113] In response to an expansion of the identified display area, the electronic device can display a screen 820 including an input UI. The electronic device can display an input UI on the expanded display area, including UI components determined based on user input. According to embodiments, the determined UI components may include an expanded text input field 821 and a keyboard 822 for inputting text. According to various embodiments, the UI components may also include at least one of a toolbar 823, a position change button 824, a mode change button 825, and an edit button 826. An input UI including UI components determined based on user input can refer to a UI expanded based on user input. The fact that the UI is expanded may include, for example, increasing the size of the selected UI component, displaying an expanded image, further displaying additional information, or further displaying at least one of additional UI components. For example, when the display area is expanded in 840, and the previously displayed text input field 812 is selected for expansion, the electronic device can display an input UI including the expanded text input field 821. As another example, the electronic device can display a screen further including a toolbar 823 for adding additional UI components to the input UI in response to the expansion of the display area.

[0114] Toolbar 823 provides a list of functions to be offered via the input UI. When a user selects at least one function from the function list, the electronic device can add at least one UI component to provide the selected function to the input UI.

[0115] The position change button 824 can be used to change the position in which the input UI is displayed. For example, when user input for selecting the position change button 824 is received while the input UI is displayed on the right side of the display of the electronic device, the electronic device can move the position in which the input UI is displayed to the left side.

[0116] The mode change button 825 can be used to switch the display mode of the input UI (e.g., landscape mode or portrait mode). For example, Figure 8 The screen 820 can be a screen on which the input UI is displayed in landscape mode, and when the mode change button 825 is selected, the electronic device can display the input UI horizontally in the area at the bottom of the screen.

[0117] The edit button 826 can be used to edit the configuration of the input UI. For example, when user input is received to select the edit button 826, the electronic device can execute an edit mode for the input UI. In edit mode, the electronic device can display at least one of a partitioning handler and a scaling / expanding handler. The electronic device can change the size of the area where the input UI is displayed, or the size of the area where UI components are displayed within the input UI, based on drag-and-drop input to at least one of the partitioning handler and the scaling handler. The electronic device can change the position of UI components arranged within the input UI based on user input in edit mode. In edit mode, the electronic device can remove UI components included in the input UI or add UI components to the input UI based on user input.

[0118] According to various embodiments, the input UI can be displayed within the extended area 830. According to another embodiment, the input UI can be displayed in another area. For example, as shown on screen 820, when the input UI is displayed in the user input for selecting the mode change button 825 and is displayed in portrait mode, the input UI can be displayed in landscape mode at the bottom of screen 820.

[0119] In 850, when the display area is expanded, even on a horizontally displayed screen 840, the electronic device can display an input UI including at least one of a text input field 841, a key area 842, and a toolbar 843.

[0120] Figure 9 This illustrates an electronic device (e.g., according to various embodiments) Figure 1 and Figure 2Electronic devices 100 Figure 3 and Figure 4 30 electronic devices Figure 5 Electronic device 101 or Figure 6 The flowchart 900 shows the process of providing a UI based on an object selected by the user input in the electronic device 101.

[0121] The electronic device can provide an input UI based on the presence or absence of interaction with the user. The electronic device can perform operation 910 of displaying a screen on at least a portion of the display area of ​​the display. According to various embodiments, the screen displayed in operation 910 may include objects related to the functions provided by the electronic device. For example, the electronic device may display a screen including at least one of the following: an input field for displaying information entered by the user, a keypad for entering text, a list of emojis for entering emojis, and a photo input icon for entering a photo.

[0122] The electronic device can perform operation 920 to determine whether the display area has expanded. When the display area is expanded, the electronic device can perform operation 930 to determine whether an object selected by user input exists. For example, the electronic device can determine that an object selected by user input exists after receiving user input on a text input area displayed on the screen or while retaining user input and recognizing the expansion of the display area. The selected object may include a UI object associated with the functionality provided by the electronic device. The UI object may include at least one of, for example, an input field for displaying input information (e.g., text, image, or emoji), a virtual keypad for inputting text, a voice reception status display object indicating whether an operation for receiving voice input is being performed, a handwriting input area for receiving handwriting input, an emoji list for selecting emojis, a thumbnail list for selecting images, and a preview image for camera capture, but the UI object is not limited to these.

[0123] When it is determined through user input that an object selected by the user is present on the screen displayed in operation 910, the electronic device can determine UI components based on the selected object. The electronic device can then perform operation 940, displaying an input UI that includes the determined UI components. According to various embodiments, the UI components included in the input UI and displayed in operation 940 can be expanded and displayed in a wider area than when displayed on the screen in operation 945. When it is determined through user input that no object selected by the user is present on the screen displayed in operation 910, the electronic device can expand and display the screen before expanding the display area.

[0124] Figure 10 This illustrates electronic devices according to various embodiments (e.g., Figure 1 and Figure 2Electronic devices 100 Figure 3 and Figure 4 30 electronic devices Figure 5 Electronic device 101 or Figure 6 A diagram illustrating an example of screen output when the display area of ​​an electronic device 101 is expanded without any interaction with the user for selecting an object.

[0125] Electronic devices according to various embodiments can display a user interface (UI) on a display. For example, the electronic device can display a screen 1010 including a virtual keypad 1011 for receiving text. When an expansion of the display area is recognized while displaying screen 1010, the electronic device can display a screen 1020 scaled in the direction of the expanded display area, while maintaining the layout of the UI including the virtual keypad 1011. For example, as... Figure 10 As shown, when the display area expands in the horizontal direction, the electronic device can expand in the horizontal direction and display the screen.

[0126] Figure 11 This illustrates electronic devices according to various embodiments (e.g., Figure 1 and Figure 2 Electronic devices 100 Figure 3 and Figure 4 30 electronic devices Figure 5 Electronic device 101 or Figure 6 The electronic device 101) provides a flowchart 1100 of the process for receiving multimodal inputs via an input UI.

[0127] The electronic device can perform the operation 1110 of displaying a screen on a display area and selecting at least one object included in the screen. For example, the electronic device can receive touch input at a location where the object is displayed. The selected object may include a UI object associated with the functions provided by the electronic device. According to an embodiment, the electronic device can select multiple UI objects based on user input. For example, the electronic device can select a voice input button for performing voice input functions and a keypad for inputting text.

[0128] The electronic device can perform operation 1120 to identify whether the display area has been expanded. When the expansion of the display area is identified, the electronic device can perform operation 1130 to display an input UI for receiving user input using an input method corresponding to one or more selected UI objects. When multiple types of input methods corresponding to one or more selected UI objects exist, the input UI may include multiple input UIs for receiving user input through multiple input methods. User input through multiple input methods can be referred to as multimodal input. For example, the electronic device may display an input UI on a display screen, which includes UI components for receiving voice input and UI components for receiving text.

[0129] exist Figure 11 Although operation 1110 is performed before operation 1120, operation 1110 can be performed after operation 1120. For example, when the display is expanded to show the input UI including the input field and keypad, user input for selecting a voice input button included in the toolbar can be received. In response to the selection of the voice input button, the electronic device can add UI components related to the operation of receiving voice input to the input UI and can display the result.

[0130] Figure 12 This illustrates an electronic device (e.g., according to various embodiments) Figure 1 and Figure 2 Electronic devices 100 Figure 3 and Figure 4 30 electronic devices Figure 5 Electronic device 101 or Figure 6 A diagram showing an example of a screen for receiving multimodal inputs displayed by an electronic device 101.

[0131] The electronic device can display a screen 1211, which includes an input field 1211 for displaying input information and a voice input button 1212. The electronic device can receive user input for selecting the input field 1211 and user input for selecting the voice input button 1212. When the display area 1230 is expanded, and both the input field 1211 and the voice input button 1212 are selected, the electronic device can display a screen 1220 including an input UI on the expanded display area. The input UI displayed on the screen 1220 may include the expanded input field 1221, a voice input-related UI 1223, and a virtual keypad 1222.

[0132] like Figure 12 As shown on screen 1220, the electronic device can provide an input UI including multiple UI components for receiving user input according to multiple input methods, thereby providing a UI that allows users to easily use multimodal input methods.

[0133] Figure 13 This illustrates the switching between electronic devices according to various embodiments (e.g., Figure 1 and Figure 2 Electronic devices 100 Figure 3 and Figure 4 30 electronic devices Figure 5 Electronic device 101 or Figure 6 An example diagram of the location of the input UI is shown in the electronic device 101.

[0134] The electronic device can display screen 1310, on which an input UI is displayed via a monitor on the right side area 1311 of the display area. The input UI may include a position change button 1330 for moving the position of the input UI displayed on the right side area 1311.

[0135] In response to user input for selecting the position change button 1330 displayed on screen 1310, the electronic device can display screen 1320, on which the input UI is set in the left area 1321. In response to user input for selecting the position change button 1330 of the input UI set in the left area 1321, the electronic device can display screen 1310 again, on which the input UI is set in the right area 1311.

[0136] Figure 14 This illustrates electronic devices according to various embodiments (e.g., Figure 1 and Figure 2 Electronic devices 100 Figure 3 and Figure 4 30 electronic devices Figure 5 Electronic device 101 or Figure 6 A flowchart 1400 shows the process by which an electronic device 101 provides an input UI based on the state of an application being executed.

[0137] Electronic devices can provide an input UI based on usage status. For example, when the application executed by the electronic device provides a landscape mode screen, the electronic device can display the input UI on the lower area of ​​the screen displayed on the display area. In this document, landscape mode can refer to an operating mode in which the image displayed on the screen is arranged downwards and displayed on a side surface of the electronic device with a longer length. Portrait mode can refer to an operating mode in which the image displayed on the screen is arranged downwards and displayed on a side surface of the electronic device with a shorter length.

[0138] The electronic device can perform operation 1410 of displaying a screen on at least a portion of the display area. The screen displayed in operation 1410 may include an execution screen that displays as the result of execution of at least one application. When displaying execution screens for multiple applications, the electronic device can divide the display area into multiple areas and display the corresponding execution screens.

[0139] The electronic device can perform operation 1420 to identify an expansion of the display area. When the expansion of the display area is identified, the electronic device can provide an input UI based on the state of the electronic device. In response to the expansion of the display area, the electronic device according to the embodiment can perform operation 1430 to determine whether information about the application displaying the execution screen meets certain conditions. For example, the electronic device can determine whether the application displaying the execution screen is a specified application (e.g., a video playback application). Another example is that the electronic device can determine whether the specified application is operating in a specified operating mode (e.g., a full-screen video playback operation state). Yet another example is that the electronic device can determine whether execution screens of multiple applications are being displayed in multiple windows.

[0140] When it is determined that the information regarding the application displayed on the execution screen does not meet the conditions, the electronic device may perform operation 1440 of displaying the input UI in the first area. For example, the electronic device may display the input UI in the area configured as the default (e.g., Figure 8 The input UI is displayed in the extended area 830. When it is determined that the information about the application displayed on the screen meets the conditions, the electronic device can perform the operation 1450 of displaying the input UI in the second area. The second area can be a region determined based on the operating state of the electronic device.

[0141] Figure 15 This illustrates an electronic device (e.g., according to various embodiments) Figure 1 and Figure 2 Electronic devices 100 Figure 3 and Figure 4 30 electronic devices Figure 5 Electronic device 101 or Figure 6 An example diagram of an electronic device 101 showing a screen for providing an input UI based on the state of an application being executed.

[0142] refer to Figure 15 When the electronic device reproduces video content with a long horizontal direction on the entire screen, the electronic device can display the screen in landscape mode. When the electronic device executes video application 1511 and the display area is expanded while the video content is being reproduced, the electronic device can display screen 1510, which displays an input UI arranged horizontally on the bottom area 1512.

[0143] For example, when an electronic device is displaying multiple application screens, it can display the input UI in the lower area of ​​the screen displayed on the main display area. (See reference) Figure 15 When electronic devices such as Figure 8As shown in screen 820, the input UI is vertically displayed on the right side of the display area. While each application execution screen is displayed in the first area 1511, the second area 1512, and the third area 1513, all or most of the execution screens displayed in the second area 1512 and the third area 1513 may not be displayed within the display area. Alternatively, when the area of ​​the application's execution screen is reduced as the input UI is displayed, the screen may become distorted or too small. Therefore, when an expansion of the display area is detected while multiple application execution screens are displayed in multiple windows, the electronic device can display screen 1520 including the input UI horizontally positioned in the bottom area 1513.

[0144] Figure 16 This illustrates electronic devices according to various embodiments (e.g., Figure 1 and Figure 2 Electronic devices 100 Figure 3 and Figure 4 30 electronic devices Figure 5 Electronic device 101 or Figure 6 The electronic device 101) provides an example of a screen with a modified input UI.

[0145] When the operating system or application providing the input UI is updated, the electronic device may change the configuration of the input field (e.g., the design, layout, or functionality provided by the input area). In electronic devices according to various embodiments, since the display area is expanded even if the configuration of the input field changes, the operation of providing the input UI can be performed in the same manner.

[0146] refer to Figure 16 The electronic device can display a screen 1610 including an input UI configured using UI components including function buttons 1611. According to various embodiments, when the layout or design of the input field changes due to updates to the electronic device's operating system or the application providing the input UI, the electronic device may include an input area 1621 where the layout or design has changed, but may similarly display a screen 1620 including the remaining UI components. When function button 1611 is changed to another function button 1631 due to updates to the electronic device's operating system or the application providing the input UI according to various embodiments, the electronic device can display a screen 1630 including an input UI that includes another function button 1631 instead of function button 1611.

[0147] Figure 17 This illustrates an electronic device (e.g., according to various embodiments) Figure 1 and Figure 2 Electronic devices 100 Figure 3 and Figure 4 30 electronic devices Figure 5 Electronic device 101 or Figure 6 An example diagram of an electronic device 101 showing a screen configured to provide an input UI based on an application executed by the electronic device.

[0148] When a third-party application includes an input field, in response to the recognition of an expansion of the display area while the third application is being executed, the electronic device can configure the input UI based on the layout provided by the input field of the third-party application.

[0149] refer to Figure 17 The electronic device can display the execution screen 1710 of the instant messaging program, including function buttons 1711 in the input field. When the expansion of the display area is recognized while the execution screen 1710 of the instant messaging program is being displayed, the electronic device can display an input UI including the function buttons 1711 included in the input field of the instant messaging program within the expanded display area.

[0150] Figure 18 This illustrates an electronic device (e.g., according to various embodiments) Figure 1 and Figure 2 Electronic devices 100 Figure 3 and Figure 4 30 electronic devices Figure 5 Electronic device 101 or Figure 6 An example diagram of an electronic device 101 showing a screen for providing an input UI in the form of a floating UI.

[0151] A floating UI refers to a UI in which input UI overlaps and is displayed on the screen, and the position of the input UI can move freely within the screen. Electronic devices can display screens including floating UIs. For an example of a floating UI, see [reference 1800]. Figure 18 The input UI 1820 may include at least one of an input field 1821, a UI component (e.g., a keypad area) 1822, a pin button 1823, a processor 1824, an input function button 1825, or a toolbar 1826. The electronic device may configure the position of the input UI 1820 to remain stationary or to move its position based on user input for selecting the pin button 1823. The electronic device may move the position of the input UI 1820 based on user input (e.g., drag and drop) for selecting and moving the processor 1824.

[0152] According to an embodiment, when multiple application execution screens are displayed, the electronic device can display an application to receive user input received via the input UI 1820, distinguishing it from other applications. For example, the periphery of the area where the application receiving user input is displayed can be displayed in a different color.

[0153] The components for UI 1820 are not limited to Figure 18 The configuration is shown. For example, input field 1821 may not be included in the input UI 1820. The electronic device may highlight and display input field 1811 included in the application's execution screen 1810, with information input via the input UI 1820 using its information. For example, an indicator may be displayed in or near the area where input field 1811 is displayed.

[0154] Figure 19 This illustrates electronic devices according to various embodiments (e.g., Figure 1 and Figure 2 Electronic devices 100 Figure 3 and Figure 4 30 electronic devices Figure 5 Electronic device 101 or Figure 6 A flowchart 1900 of the process of configuring and displaying the input UI of the electronic device 101.

[0155] The electronic device can determine the area where UI components are to be displayed based on user input. The electronic device can perform operation 1910 of displaying a screen on at least a portion of the display area. In operation 1910, the electronic device can receive user input for the displayed screen.

[0156] The electronic device can perform the operation of recognizing the expansion of the display area 1930. The electronic device can perform the operation of determining the area of ​​the display UI component based on the user input received in operation 1920 1940.

[0157] For example, in operation 1910, with the input area and keypad displayed, the electronic device can allow the display of an area corresponding to the object selected by the user input within the input area and keyboard to be displayed over a wide area. As another example, in operation 1920, the electronic device can receive a long-press input to an object displayed on the screen and determine the area where the UI component will be displayed based on the length of the time the long-press input is held (e.g., displaying in a wider area as the time period increases). The electronic device can then perform operation 1950, displaying the UI component corresponding to the user input within the determined area of ​​the input UI.

[0158] Figure 20 This illustrates an electronic device (e.g., according to various embodiments) Figure 1 and Figure 2 Electronic devices 100 Figure 3 and Figure 4 30 electronic devices Figure 5 Electronic device 101 or Figure 6A diagram showing an example of a screen displayed by an electronic device 101 for providing an input UI based on user input configuration.

[0159] refer to Figure 20 The electronic device can display a screen 2000 including an input field 2001 and a keypad 2002. When the display area is expanded while selecting the input field 2001, the electronic device can display a screen 2010 including an input UI, which is displayed on an area of ​​the input field 2011 that is wider than the keypad area 2012. Conversely, when the display area is expanded while selecting the keypad 2002, the electronic device can display a screen 2020, in which the keypad 2022 is wider than the input field 2021.

[0160] Figure 21 This illustrates an electronic device (e.g., according to various embodiments) Figure 1 and Figure 2 Electronic devices 100 Figure 3 and Figure 4 30 electronic devices Figure 5 Electronic device 101 or Figure 6 An example diagram of an electronic device 101 showing a screen for providing an input UI when the display area is expanded.

[0161] When displaying input fields (e.g., Figure 8 The screen of the input field 811 (e.g., Figure 8 When the display area is expanded while the screen 810 is being displayed, the electronic device according to various embodiments may display screen 2100 for further displaying additional UI components 2102 while the display of the input field 2101 is maintained.

[0162] Figure 22 This illustrates an electronic device (e.g., according to various embodiments) Figure 1 and Figure 2 Electronic devices 100 Figure 3 and Figure 4 30 electronic devices Figure 5 Electronic device 101 or Figure 6 An example diagram of an electronic device 101 showing a screen for providing an input UI including a voice input UI component and a handwriting input UI component.

[0163] Electronic devices can display an input UI that includes UI components for receiving user input using multiple input methods. For example, a display may include an input UI (which includes an input area, e.g., ...) Figure 8 Input area 821), toolbar (for example, Figure 8 The toolbar (823) and the keypad (e.g., Figure 8 The key area 822) of the screen (e.g., Figure 8 In the state of screen 820, the electronic device can receive user input for selecting the voice input button and handwriting input button 2241 from the function buttons included in the toolbar. In response to the selection of the voice input button and handwriting input button 2241, the electronic device can add the voice input UI area 2220 and the handwriting input area 2230 to the input UI. Therefore, the electronic device can display screen 2200 including the input UI, which includes input field 2210, voice input area 2220, handwriting input area 2230 and keypad 2215.

[0164] Figure 23 This illustrates an electronic device (e.g., according to various embodiments) Figure 1 and Figure 2 Electronic devices 100 Figure 3 and Figure 4 30 electronic devices Figure 5 Electronic device 101 or Figure 6 An example diagram of an electronic device 101 showing a screen for providing an input UI that allows input of images.

[0165] The electronic device can display a screen 2310 including an input UI when the display area is extended. According to an embodiment, the input UI may include an image input function button 2311 for providing image insertion functionality. The electronic device can receive user input for selecting the image input function button 2311.

[0166] In response to selection by the image input function button 2311, the electronic device can be configured with an input UI including an image list area 2321 that displays a list of thumbnail images stored in the electronic device for selecting the image to be input. The electronic device can display a screen 2320 including the configured input UI. When the screen 2320 is displayed, the electronic device can receive user input for selecting the camera input button 2322 to insert an image captured by the camera.

[0167] In response to selection by the camera input button 2322, the electronic device can add a preview area 2322 for displaying a preview image captured using the camera to the input UI. The addition of the preview area 2322 allows a screen 2330 to be displayed, which includes an image list area 2321 and the preview area 2322.

[0168] Figure 24 This illustrates electronic devices according to various embodiments (e.g., Figure 1 and Figure 2 Electronic devices 100 Figure 3 and Figure 4 30 electronic devices Figure 5Electronic device 101 or Figure 6 A flowchart 2400 shows the process of an electronic device 101 outputting and inputting a UI based on the expansion direction of the display area.

[0169] Electronic devices according to various embodiments can be configured to be in multiple directions (e.g., Figure 1 The display area is extended in both the +x and -x directions. An electronic device capable of extending the display area in at least one of a plurality of directions can perform operation 2410 of displaying a screen on at least a portion of the display area. The electronic device can also perform operation 2420 of recognizing the extension of the display area while displaying the screen.

[0170] When an expansion of the display area is recognized, the electronic device can perform a determination of whether the expansion direction is a first direction (e.g., Figure 1 The operation 2430 (in the +x direction) is performed. When the expansion direction is the first direction, the electronic device can perform the operation 2440 of displaying the input UI in the first area. For example, the first area can be set in the direction in which the display area expands within the screen (e.g., ...). Figure 1 The region in the +x direction. For example, the first region could be the following: where the display region is in the first direction (e.g., Figure 1 When expanding in the +x direction, the display area inserted into the housing retracts to the outside. When the expansion direction is a second direction, the electronic device can perform the operation 2445 of displaying the input UI in the second area. For example, the second area can be set in the direction in which the display area expands within the screen (e.g., ...). Figure 1 The region in the -x direction. For example, the second region could be the following: where the display region is in the second direction (e.g., Figure 1 When expanding in the -x direction, the display area inserted into the housing retracts to the outside.

[0171] Figure 25 This illustrates an electronic device (e.g., according to various embodiments) Figure 1 and Figure 2 Electronic devices 100 Figure 3 and Figure 4 30 electronic devices Figure 5 Electronic device 101 or Figure 6 An example diagram of an electronic device 101 showing a screen for providing an input UI based on the expansion direction of the display area.

[0172] According to an embodiment, an electronic device displaying a screen 2510 including an input field 2513 can expand its display area in a first direction 2511 or a second direction 2512. When the display area is expanded in the first direction 2511, the electronic device can display a screen 2520, and input UI is displayed on the screen 2520 in the first area 2521. Conversely, when the display area is expanded in the second direction 2512, the electronic device can display a screen 2530, and input UI is displayed on the screen 2530 in the second area 2531.

[0173] An electronic device according to various embodiments may include a display configured to display a screen on at least a portion of a display area, a sensor configured to detect expansion or contraction of the display area, and a processor configured to be operatively connected to the display. The processor may be configured to receive user input associated with the screen displayed on at least a portion of the display area, identify expansion of the display area via the sensor, and, in response to the receipt of user input and the identification of the expansion of the display area, display an input UI including UI components determined based on the user input on at least a portion of the expanded display area.

[0174] According to various embodiments, a screen displayed on at least a portion of the display area may include a UI object associated with functions provided by an electronic device. A processor may be configured to select a UI object based on user input, determine functions associated with the UI object, and determine UI components included in the input UI based on the determined functions. The UI components may be configured to control the display such that the UI components can be displayed on an area having a larger area than the area where the UI object is displayed.

[0175] According to various embodiments, user input may include touch input. The processor may be configured to: when an object within the screen is selected via touch input and an expansion of the display area is detected, configure an input UI based on the object; control the display to display the input UI on that portion of the display area; and when no object is selected and an expansion of the display area is detected, control the display to expand and display the screen in the direction of the expansion of the display area.

[0176] According to various embodiments, the screen may include at least one of an input field for displaying input content or a function item for performing functions of the electronic device. User input may be directed to at least one of the input field and the function item.

[0177] According to various embodiments, the screen displayed on at least a portion of the display area may include an execution screen for at least one application. The processor may be configured to display an input UI via the display based on information about at least one application.

[0178] According to various embodiments, the information about at least one application may include information about the type of at least one application or whether the execution screen of at least one application corresponds to multiple applications. The processor may be configured to determine, based on the information about at least one application, the area where the input UI will be displayed.

[0179] The processor of the electronic device according to various embodiments can be configured to determine the size of the area in which UI components are displayed within the input UI based on user input.

[0180] According to various embodiments, user input may include touch input for selecting an object associated with a UI component. The processor may be configured to determine the size of the area where the UI component is displayed based on the length of time the touch input is held.

[0181] According to various embodiments, the input UI may include multiple UI components for receiving user input via different input methods. These multiple UI components may include at least one of a keypad component for receiving text input, a voice input component for receiving voice input, or a handwriting tablet component for receiving handwriting input.

[0182] According to various embodiments, UI components included in the input UI may include a list of images for selecting images stored in an electronic device.

[0183] The electronic device, according to various embodiments, may further include a camera. UI components included in the input UI may include a preview image obtained through the camera.

[0184] Electronic devices according to various embodiments can be configured to expand or shrink a display area in a first direction or a second direction. A processor can be configured to determine the expansion direction of the display via a sensor and, based on the determined direction, determine the area in which an input UI is displayed.

[0185] A method of operating an electronic device including a display according to various embodiments may include: displaying a screen on at least a portion of a display area of ​​the display; receiving user input associated with the screen displayed on at least a portion of the display area; identifying an extension of the display area; and, in response to receiving the user input and identifying the extension of the display area, displaying an input UI including UI components determined based on the user input on at least a portion of the extended display area.

[0186] According to various embodiments, a screen displayed in at least a portion of the display area may include a UI object associated with functions provided by an electronic device. The method of operating the electronic device according to various embodiments may further include selecting a UI object based on user input, determining a function associated with the UI object, and determining UI components included in the input UI based on the determined function. Displaying the input UI may include displaying UI components on an area larger than the area where the UI object is displayed.

[0187] According to various embodiments, user input may include touch input. Displaying the input UI may include: configuring the input UI based on the object when an expansion of the display area is detected while an object is selected within the screen via touch input; displaying the input UI on a portion of the display area; and expanding and displaying the screen in the direction of the expansion of the display area when an expansion of the display area is detected without an object being selected.

[0188] According to various embodiments, a screen displayed on at least a portion of the display area may include an execution screen for at least one application. A method of operating an electronic device according to various embodiments may further include determining the type of at least one application or whether the execution screen of at least one application corresponds to multiple applications, and determining, based on the determination result, an area in which an input UI will be displayed.

[0189] According to various embodiments, user input may include touch input for selecting an object associated with a UI component included in the input UI, and displaying the input UI may include determining the area of ​​the region in which the UI component is displayed based on the length of time during which the touch input is held.

[0190] According to various embodiments, the input UI may include multiple UI components for receiving user input via different input methods.

[0191] The method of operating an electronic device according to various embodiments may further include determining the orientation of the display extension and determining an area in which an input UI is displayed based on the determined orientation.

[0192] The methods according to the various embodiments described in the claims or specification of this disclosure may be implemented in hardware, software, or a combination of hardware and software.

[0193] When the method is implemented in software, a computer-readable storage medium may be provided for storing one or more programs (software modules). The one or more programs stored in the computer-readable storage medium may be configured to be executed by one or more processors within an electronic device. The at least one program may include instructions causing the electronic device to perform the method according to the various embodiments defined by the appended claims and / or disclosed herein.

[0194] The program (software module or software) can be stored in non-volatile memory, including random access memory and flash memory, read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic disk storage devices, optical disc read-only memory (CD-ROM), digital versatile optical disc (DVD), or other types of optical storage devices or magnetic tape. Alternatively, any combination of some or all of these can form the memory storing the program. Furthermore, an electronic device may include multiple such memories.

[0195] Furthermore, the program can be stored in an attachable storage device that can be accessed by the electronic device via a communication network, such as the Internet, intranet, local area network (LAN), wide area network (WLAN), and storage area network (SAN), or a combination thereof. This storage device can access the electronic device via an external port. Additionally, a standalone storage device on the communication network can access portable electronic devices.

[0196] In the detailed embodiments described above, elements included in this disclosure are represented in a singular or plural form according to the presented detailed embodiments. However, for the sake of convenience, singular or plural forms have been suitably chosen as presented, and this disclosure is not limited to elements expressed in a singular or plural form. Thus, an element represented in a plural form may also include a single element, or an element represented in a singular form may include multiple elements.

[0197] Although specific embodiments have been described in the detailed description of this disclosure, it will be clear that various modifications and changes can be made thereto without departing from the scope of this disclosure. Therefore, the scope of this disclosure should not be defined as limited to the embodiments, but rather as defined by the appended claims and their equivalents.

Claims

1. An electronic device comprising: The housing includes a first housing portion and a second housing portion, the second housing portion being configured to movably engage with the first housing portion between a retracted position and an extended position; A flexible display, coupled to the first housing portion and the second housing portion, such that as the housing moves between the retracted position and the unfolded position, the size of the area of ​​the flexible display visible from the front of the housing changes; A mechanical structure in which a mechanical operation occurs to move the second housing portion relative to the first housing portion; At least one processor is configured to be operatively connected to the display. Wherein, the at least one processor is configured to: A screen displaying a first user interface (UI) including a UI object displayed on at least a portion of the flexible display visible from the front, the UI object being associated with the input functions of the electronic device; Receive first user input for selecting a UI object displayed on at least a portion of the flexible display; The mechanical structure alters the size of the area of ​​the flexible display visible from the front; and In response to receiving the first user input and a change in size, a second UI is displayed on at least a portion of the area where the size has changed. The second UI includes an area for receiving the second user input via an input function associated with a selected UI object.

2. The electronic device as claimed in claim 1, wherein, The screen displayed on at least a portion of the flexible display, which is visible from the front, includes UI objects associated with the input functions provided by the electronic device. The at least one processor is further configured to: Determine the input functions associated with the UI object; and The UI components to be included in the second UI are determined based on the defined input functions, and In the second UI, the UI component controls the flexible display such that the UI component is displayed on an area having a larger area than the area where the UI object is displayed.

3. The electronic device as claimed in claim 1, in, The first user input includes touch input, and The at least one processor is further configured to: When the area of ​​the flexible display visible from the front changes while a UI object is selected on the screen via the touch input, a second UI is configured based on the UI object; Controlling the flexible display to display a second UI on a portion of the flexible display visible from the front; and When the UI object is not selected, the flexible display is controlled to expand and display the screen in a direction that expands from the area of ​​the flexible display visible from the front.

4. The electronic device as claimed in claim 1, in, The screen includes at least one of an input field for displaying input content or a function item for performing input functions provided by the electronic device; and The user input is associated with at least one of the input field and the function item.

5. The electronic device as claimed in claim 1, in, The screen displayed on at least a portion of the flexible display, visible from the front, includes at least one application execution screen, and The at least one processor is also configured to control the flexible display to display a second UI based on information about the at least one application.

6. The electronic device as claimed in claim 5, in, Information regarding the at least one application includes at least one of the following: information about the type of the at least one application or information about whether the execution screen of the at least one application corresponds to multiple applications, and The at least one processor is further configured to determine the area where the second UI is to be displayed based on information about the at least one application.

7. The electronic device as claimed in claim 1, wherein, The at least one processor is also configured to determine the size of the area of ​​the flexible display visible from the front based on the first user input, wherein UI components are displayed within a second UI.

8. The electronic device as claimed in claim 7, in, The user input includes touch input for selecting a UI object related to the input function, and The at least one processor is further configured to determine the size of the area of ​​the flexible display in which the UI components are displayed, which is visible from the front, based on the length of the time during which the touch input is held.

9. The electronic device as claimed in claim 1, wherein, The second UI includes UI components that are added to the first UI or extended from a portion of the first UI.

10. The electronic device of claim 9, wherein, The second UI includes at least one of the following: a key area component for receiving text input, a voice input component for receiving voice input, or a handwriting input area component for receiving handwriting input, corresponding to the input function.

11. The electronic device as claimed in claim 1, wherein, The first UI is configured based on the first layout, and The second UI is configured based on the second layout.

12. The electronic device of claim 11, wherein, The at least one processor is further configured to determine a second layout based on a UI object associated with the first user input, and The second layout includes a UI component that includes an image list for selecting images stored in the electronic device.

13. The electronic device of claim 11, further comprising: camera, The at least one processor is further configured to determine a second layout based on a UI object associated with the first user input, and The second layout includes a UI component, which includes a preview image obtained through the camera.

14. The electronic device as claimed in claim 1, in, The area of ​​the flexible display visible from the front is configured to expand or shrink in a first direction or a second direction, and The at least one processor is further configured to: Determine the direction of expansion of the area of ​​the flexible display visible from the front; and Determine the area of ​​the flexible display in which the second UI is displayed that is visible from the front.

15. A method performed by an electronic device, the electronic device comprising: The housing includes a first housing portion and a second housing portion, the second housing portion being configured to movably engage with the first housing portion between a retracted position and an extended position; A flexible display, coupled to the first housing portion and the second housing portion, such that as the housing moves between the retracted position and the unfolded position, the size of the area of ​​the flexible display visible from the front of the housing changes; And a mechanical structure, wherein a mechanical operation occurs to move the second housing portion relative to the first housing portion. The method includes: A screen displaying a first user interface (UI) includes UI objects displayed based on a layout on at least a portion of the flexible display visible from the front, the UI objects being associated with the input functions of the electronic device; Receive first user input for selecting a UI object displayed on at least a portion of the flexible display; The mechanical structure alters the size of the area of ​​the flexible display visible from the front; and In response to receiving the first user input and a change in size, a second UI is displayed on at least a portion of the area where the size has changed. The second UI includes an area for receiving the second user input via an input function associated with a selected UI object.

Images