Screen layout changing method utilizing touch information, and multi-foldable electronic device thereof

Abstract

In various embodiments of the present invention, instructions stored in a memory (130) can instruct, when executed individually and / or collectively by a processor, a multi-foldable electronic device to: detect a folding state of a first hinge structure on the basis of information acquired from at least one sensor; identify an input means type if the first hinge structure is in the folding state; if the input means type is a pen, determine a window use size of a third display area on the basis of the pen type; if the input means type is a finger of a user, determine the window use size of the third display area on the basis of a touch area and shape; and change, on the basis of the determined window use size, a layout of a screen displayed in the third display area. Various embodiments are possible.

Description

Method for changing screen layout using touch information and the multi-foldable electronic device thereof

[0001] Various embodiments of the present disclosure disclose a method for changing the layout of a screen using touch information and a multi-foldable electronic device thereof.

[0002] With the development of digital technology, various types of electronic devices such as mobile communication terminals, PDAs (personal digital assistants), electronic notebooks, smartphones, tablet PCs (personal computers), and wearable devices are widely used. To support and enhance the functionality of these electronic devices, the hardware and / or software parts of the devices are continuously being improved. For example, electronic devices are being developed to go beyond bendable forms to include foldable displays, or displays that can be expanded in a slide or rolling format. Electronic devices including such displays can provide convenience of use depending on the state of the display and the orientation of the electronic device.

[0003] For example, electronic devices may have new form factors such as multi-display devices (e.g., dual display devices) or foldable electronic devices. Foldable electronic devices are equipped with a foldable (or bendable) display (e.g., foldable display or flexible display) and can be used by folding or unfolding. For example, in the case of foldable electronic devices, various information can be visually provided through the display (or display area) exposed in the unfolded or folded state.

[0004] The information described above may be provided as related art for the purpose of aiding understanding of the present disclosure. No claim or determination is made as to whether any of the foregoing may be applied as prior art related to the present disclosure.

[0005] In one embodiment, a method and apparatus may be disclosed for determining the window usage size of a display area of ​​a multi-foldable electronic device based on the type of a pen or the touch area of ​​a finger as an input means when a part of the multi-foldable electronic device is used in a folded state, and changing the screen layout of the display area based on the determined window usage size.

[0006] A multi-foldable electronic device (101) according to one embodiment of the present disclosure comprises: a housing including a first housing (510), a second housing (520), and a third housing (530); at least one sensor (176) disposed in at least one of the first housing to the third housing; a first hinge structure (513) disposed between the first housing and the second housing so that the first housing and the second housing fold together; a second hinge structure (515) disposed between the second housing and the third housing so that the second housing and the third housing fold together; a first display area (501) disposed corresponding to the first housing; a second display area (503) disposed between the first hinge structure and the second hinge structure corresponding to the second housing; and a third display area (505) disposed corresponding to the third housing; a memory (130) for storing instructions; and The multi-foldable electronic device may include a processor (120), and when the instructions are executed individually and / or collectively by the processor, the multi-foldable electronic device may detect the folding state of the first hinge structure based on information obtained from the at least one sensor, identify the type of input means when the first hinge structure is in a folding state, determine the window usage size of the third display area based on the type of pen when the type of input means is a pen, determine the window usage size of the third display area based on the touch area and shape when the type of input means is a user's finger, and change the layout of the screen displayed in the third display area based on the determined window usage size.

[0007] A method of operation of a multi-foldable electronic device (101) comprising a housing including a first housing (510), a second housing (520), and a third housing (530) according to one embodiment of the present disclosure, at least one sensor (176) disposed in at least one of the first housing to the third housing, a first hinge structure (513) disposed between the first housing and the second housing so that the first housing and the second housing fold together, a second hinge structure (515) disposed between the second housing and the third housing so that the second housing and the third housing fold together, a first display area (501) disposed corresponding to the first housing, a second display area (503) disposed between the first hinge structure and the second hinge structure corresponding to the second housing, and a third display area (505) disposed corresponding to the third housing, wherein the first hinge The method may include an operation to detect the folding state of the structure, an operation to identify the type of input means when the first hinge structure is in a folding state, an operation to determine the window usage size of the third display area based on the type of pen when the type of input means is a pen, an operation to determine the window usage size of the third display area based on the touch area and shape when the type of input means is a user's finger, and an operation to change the layout of the screen displayed in the third display area based on the determined window usage size.

[0008] According to one embodiment, when a part of a multi-foldable electronic device is used in a folded state, the window usage size of the display area of ​​the multi-foldable electronic device is determined based on the type of pen, which is an input means, or the touch area of ​​a finger, and by changing the screen layout of the display area based on the determined window usage size, the touch usability of the display area adjacent to the folded part can be improved.

[0009] According to one embodiment, the usability of a multi-foldable electronic device can be improved by informing the user in advance of a usable area excluding a touch-resistant area in a display area adjacent to the folded part of the multi-foldable electronic device.

[0010] According to one embodiment, user convenience can be improved by informing the user in advance of the usable area of ​​the display area based on the direction and tilt of a pen touching the display area adjacent to the folded part of the multi-foldable electronic device.

[0011] According to one embodiment, by determining the usable area of ​​the display area based on the touch area or shape of a finger touching the display area adjacent to the folded part of the multi-foldable electronic device, the usable area can be optimally utilized.

[0012] FIG. 1 is a block diagram of a multi-foldable electronic device in a network environment according to one embodiment.

[0013] FIG. 2 is a drawing illustrating a display of a multi-foldable electronic device according to one embodiment.

[0014] FIG. 3 is a drawing illustrating an example of a first type of multi-foldable electronic device according to one embodiment.

[0015] FIG. 4 is a drawing illustrating an example of a second type of multi-foldable electronic device according to one embodiment.

[0016] FIG. 5 is a diagram illustrating an example in which an edge margin area occurs according to a partial folding state of a multi-foldable electronic device according to one embodiment.

[0017] FIG. 6 is a flowchart illustrating the operation method of a multi-foldable electronic device according to one embodiment.

[0018] FIG. 7a is a drawing illustrating an example in which a second hinge structure is in an unfolded state in a multi-foldable electronic device according to one embodiment.

[0019] FIG. 7b is a drawing illustrating an example in which a second hinge structure is in a folded state in a multi-foldable electronic device according to one embodiment.

[0020] FIG. 8a is a drawing illustrating an example of guiding a designated screen mode when using a pen in a multi-foldable electronic device according to one embodiment.

[0021] FIG. 8b is a drawing illustrating an example of calculating the direction and tilt of a pen in a multi-foldable electronic device according to one embodiment.

[0022] FIG. 9 is a flowchart illustrating a method for determining the window usage size according to the type of pen of a multi-foldable electronic device according to one embodiment.

[0023] FIG. 10a is a drawing illustrating an example of determining the size of a first window according to the direction and tilt of a pen in a multi-foldable electronic device according to one embodiment.

[0024] FIG. 10b is a drawing illustrating an example of determining the size of a second window according to the direction and tilt of a pen in a multi-foldable electronic device according to one embodiment.

[0025] FIG. 10c is a drawing illustrating an example of determining the size of a third window according to the direction and tilt of a pen in a multi-foldable electronic device according to one embodiment.

[0026] FIG. 11a is a drawing illustrating an example of guiding a designated screen mode when a user uses their finger in a multi-foldable electronic device according to one embodiment.

[0027] FIG. 11b is a drawing illustrating an example of calculating the touch area and shape of a finger in a multi-foldable electronic device according to one embodiment.

[0028] FIG. 12 is a flowchart illustrating a method for determining the window usage size according to the touch area and shape of a finger of a multi-foldable electronic device according to one embodiment.

[0029] FIG. 13a is a drawing illustrating an example of determining the size of a first window used according to the touch area and shape of a finger in a multi-foldable electronic device according to one embodiment.

[0030] FIG. 13b is a drawing illustrating an example of processing a touch according to the touch area and touch time of a finger in a multi-foldable electronic device according to one embodiment.

[0031] FIG. 14a is a drawing illustrating an example of processing a touch according to the touch area and touch time of a finger in a multi-foldable electronic device according to one embodiment.

[0032] FIG. 14b is a diagram illustrating an example of detecting a touch in a multi-foldable electronic device according to one embodiment and ignoring processing due to a state change.

[0033] FIG. 1 is a block diagram of a multi-foldable electronic device (101) in a network environment (100) according to various embodiments.

[0034] Referring to FIG. 1, in a network environment (100), a multi-foldable electronic device (101) may communicate with an electronic device (102) through a first network (198) (e.g., a short-range wireless communication network) or with at least one of an electronic device (104) or a server (108) through a second network (199) (e.g., a long-range wireless communication network). According to one embodiment, the multi-foldable electronic device (101) may communicate with an electronic device (104) through a server (108). According to one embodiment, a multi-foldable electronic device (101) may include a processor (120), memory (130), input module (150), sound output module (155), display module (160), audio module (170), sensor module (176), interface (177), connection terminal (178), haptic module (179), camera module (180), power management module (188), battery (189), communication module (190), subscriber identification module (196), or antenna module (197). In some embodiments, at least one of these components (e.g., connection terminal (178)) may be omitted from the multi-foldable electronic device (101), or one or more other components may be added. In some embodiments, some of these components (e.g., sensor module (176), camera module (180), or antenna module (197)) may be integrated into a single component (e.g., display module (160)).

[0035] The processor (120) can, for example, execute software (e.g., program (140)) to control at least one other component (e.g., hardware or software component) of the multi-foldable electronic device (101) connected to the processor (120) and can perform various data processing or operations. According to one embodiment, as at least part of the data processing or operations, the processor (120) can store commands or data received from other components (e.g., sensor module (176) or communication module (190)) in volatile memory (132), process the commands or data stored in volatile memory (132), and store the resulting data in non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or an auxiliary processor (123) that can operate independently or together with it (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor). For example, if a multi-foldable electronic device (101) includes a main processor (121) and an auxiliary processor (123), the auxiliary processor (123) may be configured to use lower power than the main processor (121) or to be specialized for a specified function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as part thereof.

[0036] The auxiliary processor (123) may control at least some of the functions or states associated with at least one component of the multi-foldable electronic device (101) (e.g., display module (160), sensor module (176), or communication module (190)) on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. According to one embodiment, the auxiliary processor (123) (e.g., image signal processor or communication processor) may be implemented as part of another functionally related component (e.g., camera module (180) or communication module (190)). According to one embodiment, the auxiliary processor (123) (e.g., neural network processing unit) may include a hardware structure specialized for processing an artificial intelligence model. The artificial intelligence model may be generated through machine learning. Such learning may be performed, for example, on the multi-foldable electronic device (101) itself where the artificial intelligence model is executed, or through a separate server (e.g., server (108)). The learning algorithm may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model may include a plurality of artificial neural network layers.An artificial neural network may be a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to the hardware structure, the artificial intelligence model may include a software structure, either additionally or substantially.

[0037] The memory (130) can store various data used by at least one component (e.g., processor (120) or sensor module (176)) of the multi-foldable electronic device (101). The data may include, for example, software (e.g., program (140)) and input data or output data for related commands. The memory (130) may include volatile memory (132) or non-volatile memory (134).

[0038] The program (140) may be stored as software in memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0039] The input module (150) can receive commands or data to be used for a component (e.g., processor (120)) of the multi-foldable electronic device (101) from outside the multi-foldable electronic device (101) (e.g., user). The input module (150) may include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0040] The sound output module (155) can output a sound signal to the outside of the multi-foldable electronic device (101). The sound output module (155) may include, for example, a speaker or a receiver. The speaker may be used for general purposes, such as multimedia playback or recording playback. The receiver may be used to receive incoming calls. According to one embodiment, the receiver may be implemented separately from the speaker or as part thereof.

[0041] The display module (160) can visually provide information to the outside (e.g., user) of the multi-foldable electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling said device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of the force generated by said touch.

[0042] The audio module (170) can convert sound into an electrical signal or, conversely, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150) or output sound through the sound output module (155) or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphones) that is directly or wirelessly connected to the multi-foldable electronic device (101).

[0043] The sensor module (176) can detect the operating state (e.g., power or temperature) of the multi-foldable electronic device (101) or the external environmental state (e.g., user state) and generate an electrical signal or data value corresponding to the detected state. According to one embodiment, the sensor module (176) may include, for example, a gesture sensor, a gyroscope sensor, a barometric pressure sensor, a magnetic sensor, an accelerometer sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biosensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0044] The interface (177) may support one or more specified protocols that can be used for the multi-foldable electronic device (101) to be connected directly or wirelessly to an external electronic device (e.g., electronic device (102)). According to one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0045] The connection terminal (178) may include a connector through which the multi-foldable electronic device (101) can be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0046] The haptic module (179) can convert an electrical signal into a mechanical stimulus (e.g., vibration or movement) or an electrical stimulus that the user can perceive through tactile or kinesthetic senses. According to one embodiment, the haptic module (179) may include, for example, a motor, a piezoelectric element, or an electric stimulation device.

[0047] The camera module (180) can capture still images and video. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0048] The power management module (188) can manage the power supplied to the multi-foldable electronic device (101). According to one embodiment, the power management module (188) can be implemented, for example, as at least part of a power management integrated circuit (PMIC).

[0049] The battery (189) can supply power to at least one component of the multi-foldable electronic device (101). According to one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0050] The communication module (190) can support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between a multi-foldable electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may include one or more communication processors that operate independently of the processor (120) (e.g., application processor) and support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., cellular communication module, short-range wireless communication module, or GNSS (global navigation satellite system) communication module) or a wired communication module (194) (e.g., LAN (local area network) communication module, or power line communication module). The corresponding communication module among these communication modules can communicate with an external electronic device (104) through a first network (198) (e.g., a short-range communication network such as Bluetooth, WiFi (wireless fidelity) direct, or IrDA (infrared data association)) or a second network (199) (e.g., a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules may be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can identify or authenticate a multi-foldable electronic device (101) within a communication network such as the first network (198) or the second network (199) using subscriber information (e.g., International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module (196).

[0051] The wireless communication module (192) can support 5G networks and next-generation communication technologies following 4G networks, for example, new radio access technology. NR access technology can support high-speed transmission of high-capacity data (enhanced mobile broadband (eMBB)), minimization of terminal power and connection of multiple terminals (massive machine type communications (mMTC)), or high reliability and low latency (ultra-reliable and low-latency communications (URLLC)). The wireless communication module (192) can support a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate, for example. The wireless communication module (192) can support various technologies for securing performance in the high-frequency band, such as beamforming, massive MIMO (multiple-input and multiple-output), full-dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large-scale antenna. The wireless communication module (192) can support various requirements specified in the multi-foldable electronic device (101), external electronic device (e.g., electronic device (104)), or network system (e.g., second network (199)). According to one embodiment, the wireless communication module (192) can support a Peak data rate (e.g., 20 Gbps or more) for realizing eMBB, loss coverage (e.g., 164 dB or less) for realizing mMTC, or U-plane latency (e.g., downlink (DL) and uplink (UL) each 0.5 ms or less, or round trip 1 ms or less) for realizing URLLC.

[0052] An antenna module (197) can transmit a signal or power to or from an external source (e.g., an external electronic device). According to one embodiment, the antenna module (197) may include an antenna comprising a radiator made of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). According to one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as a first network (198) or a second network (199), may be selected from the plurality of antennas, for example, by a communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device through the selected at least one antenna. According to some embodiments, in addition to the radiator, other components (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as part of the antenna module (197).

[0053] According to various embodiments, the antenna module (197) may form a mmWave antenna module. According to one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent to a first surface (e.g., bottom surface) of the printed circuit board and capable of supporting a specified high frequency band (e.g., mmWave band), and a plurality of antennas (e.g., array antennas) disposed on or adjacent to a second surface (e.g., top surface or side surface) of the printed circuit board and capable of transmitting or receiving a signal of the specified high frequency band.

[0054] At least some of the above components can be connected to each other via a communication method between peripheral devices (e.g., bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)) and exchange signals (e.g., commands or data) with each other.

[0055] According to one embodiment, commands or data may be transmitted or received between the multi-foldable electronic device (101) and an external electronic device (104) through a server (108) connected to a second network (199). Each of the external electronic devices (102, or 104) may be a device of the same or different type as the multi-foldable electronic device (101). According to one embodiment, all or part of the operations performed on the multi-foldable electronic device (101) may be performed on one or more of the external electronic devices (102, 104, or 108). For example, if the multi-foldable electronic device (101) needs to perform a function or service automatically or in response to a request from a user or another device, the multi-foldable electronic device (101) may request one or more external electronic devices to perform at least part of the function or service instead of performing the function or service itself or additionally. One or more external electronic devices that receive the above request may execute at least part of the requested function or service, or additional function or service related to the request, and transmit the result of the execution to the multi-foldable electronic device (101). The multi-foldable electronic device (101) may provide the result as is or additionally processed as at least part of the response to the request. For this purpose, for example, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used. The multi-foldable electronic device (101) may provide ultra-low latency services using, for example, distributed computing or mobile edge computing. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server using machine learning and / or neural networks.According to one embodiment, an external electronic device (104) or server (108) may be included within the second network (199). The multi-foldable electronic device (101) may be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0056] FIG. 2 is a drawing illustrating a display of a multi-foldable electronic device according to one embodiment.

[0057] Referring to FIG. 2, a display (e.g., display module (160) of FIG. 1) of a multi-foldable electronic device (e.g., multi-foldable electronic device (101) of FIG. 1) according to one embodiment may include a display having a fixed shape and / or a deformable display such as a foldable display or a rollable (or slideable) display. The components, their relationships, and their functions illustrated in FIG. 2 are merely illustrative and are not intended to limit the implementations described or claimed herein.

[0058] The display module (160) may include components such as a display panel (210), a display driver integrated circuitry (DDI) (230) (or a display driver circuitry (230)), and / or a touch circuitry (250). These components are merely exemplary. For example, the display module (160) may include other components (e.g., a circuitry for controlling a digitizer and / or a sensor (270)). For example, some components may be omitted from the display module (160).

[0059] The display panel (210) may include a plurality of pixels and a plurality of thin film transistors (TFTs) that control the plurality of pixels. For example, the plurality of TFTs may include p-channel metal-oxide semiconductor (PMOS) transistors and / or n-channel metal-oxide semiconductor (NMOS) transistors. Each of the plurality of pixels may include a set of subpixels (e.g., a subpixel for providing red light, a subpixel for providing blue light, a subpixel for providing green light, and / or a subpixel for providing white light). For example, the plurality of pixels in the display panel (210) may be driven based on a voltage (or current) provided to the TFTs through (or from) a display driving circuit (230).

[0060] A display driving circuit (230) can provide visual information (or a screen) (or one or more images) through a display panel (210) (or through a display area of ​​the display panel (210)) based on image data received from a processor (120) and / or a command for controlling the operation of subcomponents of the display driving circuit (230). The display driving circuit (230) may include subcomponents such as an interface controller (231) (e.g., including an interface control circuit), a timing controller (232) (e.g., including a timing control circuit), a command controller (233) (e.g., including a command control circuit), a GRAM (graphics random access memory) controller (234) (e.g., including a GRAM control circuit), a GRAM (235), a source driver (238), and / or a gate driver (239). The display driving circuit (230) may be configured to receive data (e.g., frame data) from the processor (120) and to control the display panel (210) to display visual information using the data. The display driving circuit (230) may be described as a display peripheral.

[0061] The above sub-components are exemplary. For example, the display driving circuit (230) may further include other sub-components (e.g., a self-drawing engine (or any suitable circuit for self-drawing)). For example, some sub-components (e.g., a GRAM controller (234) and a GRAM (235)) may be omitted from the display driving circuit (230). For example, some sub-components (e.g., a source driver (238), a gate driver (239)) may be placed as components separate from the display driving circuit (230).

[0062] The interface controller (231) may provide image data obtained from the processor (120) (e.g., display controller) to the GRAM (235) and provide commands obtained from the processor (120) to the command controller (233). For example, the interface controller (231) may be used for one or more interfaces (e.g., MIPI (mobile industry processor interface), MDDI (mobile display digital interface), SPI (serial peripheral interface), I2C (inter-integrated circuit), I3C (improved inter-integrated circuit), and / or CDP (compact display port)). In one embodiment, the image data may be stored in the GRAM (235). In one embodiment, storing the image data in the GRAM (235) may be bypassed. If storing the image data is bypassed, the image data may be provided to an image processing circuit (not shown) or source driver (238) within the display driving circuit (230) via the interface controller (231).

[0063] The timing controller (232) may provide a synchronization signal (or timing signal) to the GRAM controller (234), the source driver (238), and / or the gate driver (239). In one embodiment, the synchronization signal is generated by the timing controller (232), and the synchronization signal generated by the timing controller (232) may be provided from the timing controller (232) to the GRAM controller (234), the source driver (238), the gate driver (239), and / or the touch circuit (250). In one embodiment, the synchronization signal is generated by a synchronization signal generating circuit located outside the display driving circuit (230), and may be provided from the synchronization signal generating circuit to the timing controller (232).

[0064] The synchronization signal provided from the synchronization signal generation circuit may be provided from the timing controller (232) to the GRAM controller (234), the source driver (238), and / or the gate driver (239). For example, the synchronization signal may include a display synchronization signal (e.g., a display vertical synchronization signal and a display horizontal synchronization signal). For example, the display vertical synchronization signal and the display horizontal synchronization signal may be used for reference timing to switch the source driver (238) and the gate driver (239), respectively. For example, the synchronization signal may include a touch synchronization signal (e.g., a touch vertical synchronization signal and a touch horizontal synchronization signal). For example, the touch vertical synchronization signal and the touch horizontal synchronization signal may each be provided to the touch circuit (250). As an example, but not limited to, the frequency of the display synchronization signal may be different from the frequency of the touch synchronization signal.

[0065] The command controller (233) can provide the command to the GRAM controller (234) and / or the timing controller (232).

[0066] The GRAM controller (234) can provide the image data recorded in the GRAM (235) to the source driver (238) by scanning the image data based on the synchronization signal obtained from the timing controller (232) and the command obtained from the command controller (233). In one embodiment, before the image data is provided to the source driver (238), it can be processed based on the command provided to the image processing circuit from the command controller (233) through an image processing circuit (not shown) located between the GRAM (235) and the source driver (238).

[0067] The source driver (238) can provide color through a set of subpixels based on the display vertical synchronization signal and the image data. For example, the source driver (238) can provide a data voltage corresponding to the input image data to a plurality of pixels.

[0068] The gate driver (239) can turn on or turn off the set of subpixels based on the display horizontal synchronization signal and the light emission signal.

[0069] The touch circuit (250) may include a touch sensor controller (251) and a touch sensor (252).

[0070] A touch sensor controller (251) can control a touch sensor (252) based on a touch synchronization signal (e.g., a touch vertical synchronization signal and / or the touch horizontal synchronization signal) to obtain information about an input on a display panel (210) (e.g., a touch input or a hovering input on the display panel (210)). The touch sensor controller (251) can provide the information obtained based on the touch synchronization signal to a processor (120) or a display driving circuit (230).

[0071] The touch sensor (252) may be positioned in relation to the display panel (210). For example, the touch sensor (252) may be positioned within the display panel (210) or on the display panel (210).

[0072] FIG. 3 is a drawing illustrating an example of a first type of multi-foldable electronic device according to one embodiment.

[0073] Referring to FIG. 3, an example of changing the shape (e.g., folding state) of a display (e.g., display module (160) of FIG. 1) in a multi-foldable electronic device (e.g., multi-foldable electronic device (101) of FIG. 1) according to one embodiment may be shown. The multi-foldable electronic device (101) of FIG. 3 may represent an example of an in-folding type multi-foldable electronic device (e.g., G-type foldable electronic device).

[0074] The foldable electronic device (101) may further include at least one sensor (e.g., the sensor module (176) of FIG. 1). For example, the foldable electronic device (101) may include a first sensor module in a first housing (315), a second sensor module in a second housing (325), and a third sensor module in a third housing (335). The first to third sensor modules may include at least one of an accelerometer, a gyroscope, a geomagnetic sensor, a proximity sensor, a gesture sensor, or a Hall sensor. An accelerometer is a sensor that detects speed, and a gyroscope sensor can detect angular velocity, which is the rotational speed of an object. A geomagnetic sensor is a sensor that detects geomagnetism and can detect geomagnetic directions (e.g., azimuth) such as east, west, south, and north, like a compass. A proximity sensor detects whether an object is close, and a gesture sensor can detect infrared light. A Hall sensor can detect changes in electrical signals based on the proximity or distance of a magnetic object. The Hall sensor can detect changes in the state of the foldable electronic device (101) in an analog or digital manner.

[0075] According to one embodiment, when the multi-foldable electronic device (101) is in a folded state, at least a portion of the second display area (320) and the third display area (330) of the display module (160) are in contact with each other based on a second point (or second folding axis) (e.g., B axis) that is folded (e.g., in-folding), and at least a portion of the first display area (310) of the display module (160) and the third portion (e.g., third housing (335)) of the housing (e.g., cover) are in contact with each other based on a first point (or first folding axis) (e.g., A axis), so that they are in a closed state.

[0076] A foldable electronic device (101) may include a first hinge structure disposed between the first housing (315) and the second housing (325) so that the first housing (315) and the second housing (325) fold together, and a second hinge structure disposed between the second housing (325) and the third housing (335) so that the second housing (325) and the third housing (335) fold together. The foldable electronic device (101) may determine the angle of the first hinge structure and the second hinge structure based on information obtained through the at least one sensor (e.g., a first sensor module, a second sensor module, or a third sensor module). A foldable electronic device (101) can be determined to be in a folded state when the angle of the first hinge structure is less than the first angle (e.g., 150 degrees, 120 degrees), and can be determined to be in a folded state when the angle of the second hinge structure is less than the second angle (e.g., 150 degrees, 120 degrees). The state of the foldable electronic device (101) (e.g., unfolded state, folded state, intermediate state) can be detected.

[0077] When the foldable electronic device (101) is positioned as the front (200), the x-axis of the first sensor module or the second sensor module may represent the left / right direction, the y-axis may represent the front / back direction, and the z-axis may represent the height direction. When the foldable electronic device (101) is positioned as the front (200), the sensing data (or sensing values, sensing angles) of the x-axis, y-axis, and z-axis measured (or acquired) by the first sensor module or the second sensor module may be the same or similar. For example, since the x-axis and y-axis of the inertial sensor are not affected by the direction of gravity, and the z-axis is affected by the direction of gravity, the z-axis sensing data may be different when the foldable electronic device (101) is positioned as the front (200) and when the foldable electronic device (101) is positioned as the back (250).

[0078] According to one embodiment, when the multi-foldable electronic device (101) is in an unfolded state (e.g., an unfolded state based on the shape of FIG. 3), all display areas of the display module (160) (e.g., a first display area (310), a second display area (320), a third display area (330)) are provided as one side (or the entire side), so that the display module (160) can be used with a relatively large size. According to one embodiment, when the multi-foldable electronic device (101) is partially unfolded (e.g., an intermediate state), at least one display area of ​​the display module (160) can be provided as an area for displaying visual information (or a screen) (or one or more images). The foldable electronic device (101) can be determined to be in an unfolded state when the angle of the first hinge structure is a specified angle (e.g., 180 degrees), and can be determined to be in an unfolded state when the angle of the second hinge structure is a specified angle (e.g., 180 degrees).

[0079] In FIG. 3, the display module (160) of the multi-foldable electronic device (101) may display a folding state corresponding to a partially folded state (e.g., intermediate state) with respect to a folding axis (or hinge axis or hinge structure) (e.g., first folding axis and second folding axis) at an angle less than a certain angle (e.g., about 180 degrees).

[0080] A multi-foldable electronic device (101) may include a housing (350) (or multi-foldable housing) (e.g., a first housing (315), a second housing (325), and a third housing (335)) that fixes a display module (160) including a first display area (310), a second display area (320), and a third display area (330). According to one embodiment, the housing (350) may include a foldable structure (e.g., a hinge structure). When the multi-foldable electronic device (101) is in a folded state, the first part (301) and the third part (305) may face in opposite directions to the second part (303), and the first part (301) and the third part (305) may face in the same direction. When the electronic device (101) is in an unfolded state, the first part (301), the second part (303), and the third part (305) may be formed to face in the same direction.

[0081] According to one embodiment, the multi-foldable electronic device (101) may be an example comprising two folding axes (e.g., a first folding axis and a second folding axis) (e.g., an A-axis and a B-axis). The two folding axes (A-axis and B-axis) may each be an example employed to divide the display module (160) into three parts. The multi-foldable electronic device (101) may be folded, unfolded, or bent along the folding axes (A-axis and B-axis). The first display area (310), the second display area (320), and the third display area (330) of the display module (160) of the multi-foldable electronic device (101) may be a form in which they are folded inward (e.g., an in-folding method) so that they are not exposed to the outside of the electronic device (101). The multi-foldable electronic device (101) may represent an example of a G-type foldable electronic device, and may represent an intermediate state (e.g., partially folded state or partially unfolded state) viewed from the front of the multi-foldable electronic device (101) and a folded state viewed from the back (or bottom) of the multi-foldable electronic device (101).

[0082] In the form of the multi-foldable electronic device (101), the multi-foldable electronic device (101) being completely folded (e.g., in a folded state or in a folded state) may mean that any two parts included in the display module (160) of the multi-foldable electronic device (101) (e.g., a second display area (320) and a third display area (330)) face each other, so that the two parts become completely parallel or nearly parallel, and any part of the display module (160) (e.g., a first display area (310)) and any part included in the housing (350) (e.g., a third housing (335)) face each other, so that the two parts become completely parallel or nearly parallel. For example, the multi-foldable electronic device (101) being completely folded may mean that the two parts of the multi-foldable electronic device (101) do not necessarily have to be in contact, but are positioned in nearly close proximity.

[0083] In the form of a multi-foldable electronic device (101), the multi-foldable electronic device (101) being fully unfolded (e.g., unfolded state or unfolded state) may indicate a state in which the first display area (310), the second display area (320) of the display module (160) of the electronic device (101), and the third display area (330) of the display module (160) are visually exposed to the outside and form a flat plane like a single display module (160), and the area of ​​the display module (160) exposed to the outside is the largest or approaches the largest area.

[0084] The multi-foldable electronic device (101) may have different directions of folding or bending with respect to each folding axis (A-axis, B-axis). This is exemplary, and the multi-foldable electronic device (101) may have the same direction of folding or bending with respect to each folding axis (A-axis, B-axis). According to one embodiment, the multi-foldable electronic device (101) may have the first display area (310) and the second display area (320) of the display module (160) folded facing each other, and the second display area (320) and the third display area (330) folded facing each other. This is not limited thereto, and the multi-foldable electronic device (101) exemplified in FIG. 3 may include a form in which it is folded in an out-folding manner.

[0085] According to one embodiment, depending on the position where two folding axes (A-axis, B-axis) are employed on the multi-foldable electronic device (101), the multi-foldable electronic device (101) may be folded or bent asymmetrically with respect to each folding axis (A-axis, B-axis), and even when the multi-foldable electronic device (101) is completely folded with respect to the folding axis (A-axis, B-axis), each display area (310, 320, 330) of the multi-foldable electronic device (101) separated by the folding axes (A-axis, B-axis) may not completely overlap. According to one embodiment, even when the folding axis (A-axis, B-axis) is provided on the multi-foldable electronic device (101), a display module (160) may be employed on the front and / or rear of the multi-foldable electronic device (101), and the display module (160) may be activated or deactivated.

[0086] According to one embodiment, the multi-foldable electronic device (101) can detect the folding state (or degree of folding) (e.g., folded state, intermediate state, or unfolded state) of the multi-foldable electronic device (101). According to one embodiment, the multi-foldable electronic device (101) can detect the folding state based on a first folding angle based on a folding axis of a first hinge structure (e.g., a first folding axis or an A-axis) and a second folding angle based on a folding axis of a second hinge structure (e.g., a second folding axis or a B-axis). According to one embodiment, the multi-foldable electronic device (101) can detect the folding state and enable or disable at least some display area of ​​the display module (160).

[0087] According to one embodiment, when the multi-foldable electronic device (101) detects a folded state, all display areas of the display module (160) (e.g., a first display area (310), a second display area (320), and a third display area (330)) may be deactivated. According to one embodiment, when the multi-foldable electronic device (101) detects a folded state, the display area used by the multi-foldable electronic device (101) (e.g., a first display area (310)) may be activated, and the display area not used (e.g., a second display area (320) and a third display area (330)) may be deactivated. According to one embodiment, when the multi-foldable electronic device (101) detects an intermediate state, it can activate / deactivate at least one display area (e.g., a first display area (310), a second display area (320) and / or a third display area (330)) of the display module (160) according to the folding state of the electronic device (101).

[0088] FIG. 4 is a drawing illustrating an example of a second type of multi-foldable electronic device according to one embodiment.

[0089] Referring to FIG. 4, a multi-foldable electronic device according to one embodiment (e.g., the multi-foldable electronic device (101) of FIG. 1) may represent an example of an in-folding type multi-foldable electronic device (e.g., a C-type foldable electronic device). When the multi-foldable electronic device (101) is in a folded state, at least a portion of the first display area (410) and the second display area (420) of the display module (160) are in contact with each other based on a first point (or first folding axis) (e.g., A-axis) that is folded (e.g., in-folding), and at least a portion of the third display area (430) and the second display area (420) of the display module (160) are in contact with each other based on a second point (or second folding axis) (e.g., B-axis), so that the device is in a closed state.

[0090] When the multi-foldable electronic device (101) is in an unfolded state, all display areas of the display module (160) (e.g., first display area (410), second display area (420), third display area (430)) are provided as one side (or the entire side), allowing the display module (160) to be used in a relatively large size. According to one embodiment, when the multi-foldable electronic device (101) is partially unfolded (e.g., intermediate state), at least one display area of ​​the display module (160) may be provided as an area for displaying visual information (or a screen) (or one or more images). The display module (160) of the multi-foldable electronic device (101) may display a folding state corresponding to a partially folded state (e.g., intermediate state) at an angle less than a certain angle (e.g., about 180 degrees) with respect to a folding axis (or hinge axis or hinge structure) (e.g., first folding axis and second folding axis).

[0091] A multi-foldable electronic device (101) may include a housing (450) (or multi-foldable housing) (e.g., a first housing (415), a second housing (425), and a third housing (435)) that fixes a display module (160) including a first display area (410), a second display area (420), and a third display area (430). According to one embodiment, the housing (450) may include a foldable structure (e.g., a hinge structure). When the multi-foldable electronic device (101) is in a folded state, the first part (401) and the third part (405) may face in opposite directions to the second part (403), and the first part (401) and the third part (405) may face in the same direction. When the multi-foldable electronic device (101) is in an unfolded state, the first part (401), the second part (403), and the third part (405) may be formed to face in the same direction.

[0092] According to one embodiment, an example of a multi-foldable electronic device (101) comprising two folding axes (e.g., a first folding axis and a second folding axis) (e.g., an A-axis and a B-axis) may be shown. The two folding axes (A-axis and B-axis) may each be employed to divide the display module (160) into three parts. The electronic device (101) may be folded, unfolded, or bent along the folding axes (A-axis and B-axis). The first display area (410), the second display area (420), and the third display area (430) of the display module (160) of the multi-foldable electronic device (101) may be folded inward (e.g., in-folding method) so that they are not exposed to the outside of the electronic device (101). In FIG. 4, the multi-foldable electronic device (101) may represent an example of a C-type foldable electronic device, and may represent an intermediate state (e.g., partially folded state or partially unfolded state) viewed from the front of the multi-foldable electronic device (101) and a folded state viewed from the back (or bottom) of the multi-foldable electronic device (101).

[0093] According to one embodiment, in the form of the multi-foldable electronic device (101) illustrated in FIG. 4, the electronic device (101) being completely folded (e.g., in a folded state or in a folded state) may mean that all parts included in the display module (160) of the multi-foldable electronic device (101) (e.g., a first display area (410) and a second display area (420), and a second display area (420) and a third display area (430)) are facing each other (or are facing each other), so that the two parts are completely parallel or nearly parallel. For example, the multi-foldable electronic device (101) being completely folded may mean that the two parts of the multi-foldable electronic device (101) do not necessarily have to be in contact, but are positioned nearly close to each other.

[0094] According to one embodiment, in the form of the multi-foldable electronic device (101) illustrated in FIG. 4, the multi-foldable electronic device (101) being fully unfolded (e.g., unfolded state or unfolded state) may indicate a state in which the first display area (410), the second display area (420) of the display module (160) of the multi-foldable electronic device (101), and the third display area (430) of the display module (160) are visually exposed to the outside and form a flat plane like a single display module (160), and may indicate when the area of ​​the display module (160) exposed to the outside is the largest or approaches the largest area.

[0095] As illustrated in FIG. 4, the multi-foldable electronic device (101) may have different directions of folding or bending with respect to each folding axis (A-axis, B-axis). This is exemplary, and the multi-foldable electronic device (101) may have the same direction of folding or bending with respect to each folding axis (A-axis, B-axis). According to one embodiment, the multi-foldable electronic device (101) may be folded such that the first display area (410) and the second display area (420) of the display module (160) face each other, and the second display area (420) and the third display area (430) face each other. This is not limited to, but if the multi-foldable electronic device (101) exemplified in FIG. 4 is in a form that folds in an out-folding manner, the first display area (410), the second display area (420), and the third display area (430) are exposed to the outside, and the back surface of the first display area (410) (e.g., the first housing (415)) and the back surface of the second display area (420) (e.g., the second housing (425)) are folded facing each other, and the back surface of the second display area (420) (e.g., the second housing (425)) and the back surface of the third display area (430) (e.g., the third housing (435)) are folded facing each other.

[0096] According to one embodiment, depending on the position where two folding axes (A-axis, B-axis) are employed on the multi-foldable electronic device (101), the multi-foldable electronic device (101) may be folded or bent asymmetrically with respect to each folding axis (A-axis, B-axis), and even when the multi-foldable electronic device (101) is completely folded with respect to the folding axis (A-axis, B-axis), each display area (410, 420, 430) of the multi-foldable electronic device (101) separated by the folding axes (A-axis, B-axis) may not completely overlap. According to one embodiment, even when the folding axis (A-axis, B-axis) is provided in the multi-foldable electronic device (101) as exemplified in FIG. 4, a display module (160) may be employed on the front and / or rear of the electronic device (101), and the display module (160) may be activated or deactivated in a similar manner as described in the description section with reference to FIG. 3 above.

[0097] According to one embodiment, a multi-foldable electronic device (101) can detect a change in the shape (e.g., folding or unfolding) of a display module (160) based on various methods.

[0098] According to one embodiment, the multi-foldable electronic device (101) may include a state sensing sensor and / or an angle sensing sensor based on at least one sensor (e.g., sensor module (176) of FIG. 1). According to one embodiment, the multi-foldable electronic device (101) may determine the posture (or operation mode) (e.g., portrait mode or landscape mode) of the multi-foldable electronic device (101) based on first sensor data sensed by the state sensing sensor. According to one embodiment, the multi-foldable electronic device (101) may determine the folding angle (or folding state) of the multi-foldable electronic device (101) based on second sensor data sensed by the angle sensing sensor. In one embodiment, the posture of the multi-foldable electronic device (101) may include a portrait mode or a landscape mode used in an operation mode (e.g., phone mode or tablet mode). In one embodiment, the portrait mode may be a mode in which the multi-foldable electronic device (101) is used in portrait mode in an unfolded state (e.g., fully unfolded state or intermediate state). In one embodiment, the landscape mode may be a mode in which the multi-foldable electronic device (101) is used in landscape mode in an unfolded state (e.g., fully unfolded state or intermediate state). According to one embodiment, the multi-foldable electronic device (101) may be changed from portrait mode to landscape mode or from landscape mode to portrait mode based on a change in the orientation (e.g., rotation) of the multi-foldable electronic device (101).

[0099] According to one embodiment, a multi-foldable electronic device (101) may include a first sensor (e.g., a first angle sensing sensor) that detects a first folding angle of a first folding axis (or a first hinge structure) and a second sensor (e.g., a second angle sensing sensor) that detects a second folding angle of a second folding axis (or a second hinge structure). For example, the multi-foldable electronic device (101) can determine a first folding angle between the first housing (315, 415) and the second housing (325, 425) (or between the first display area (310, 410) and the second display area (320, 420)) based on a first folding axis (e.g., A axis) based on sensor data sensed by the first sensor. For example, the multi-foldable electronic device (101) can determine a second folding angle between the second housing (325, 425) and the third housing (335, 435) (or between the second display area (320, 420) and the third display area (330, 430)) based on a second folding axis (e.g., B axis) based on sensor data sensed by the second sensor.

[0100] According to one embodiment, the state sensing sensor may include at least one of a grip sensor, an inertial sensor, a proximity sensor, an illuminance sensor, a magnetic sensor, a Hall sensor, a gesture sensor, a bending sensor, an infrared sensor, a touch sensor, a pressure sensor, or an infrared camera, or a combination thereof. According to one embodiment, the angle sensing sensor may include at least one of an inertial sensor, a Hall sensor, a magnetic sensor, or a bending sensor, or a combination thereof. According to one embodiment, an angle sensing sensor may be positioned on one side of the electronic device (101) (e.g., a folding axis, a housing end, a bottom of a display module (160) (e.g., under a panel, and / or a bezel of a display module (160)) to measure the folding angle of the multi-foldable electronic device (101). The folding angle may represent the angle formed by two display areas separated by the folding axis of the electronic device (101) with the folding axis. According to one embodiment, the multi-foldable electronic device (101) may measure the folding angle to determine whether the electronic device (101) is in a fully folded state, a fully unfolded state, or an intermediate state in which it is unfolded (or folded) at a certain angle.

[0101] FIG. 5 is a diagram illustrating an example in which an edge margin area occurs according to a partial folding state of a multi-foldable electronic device according to one embodiment.

[0102] Referring to FIG. 5, a multi-foldable electronic device according to one embodiment (e.g., the multi-foldable electronic device (101) of FIG. 1) may be folded, unfolded, or bent based on a first hinge structure (513) or a second hinge structure (515) corresponding to two folding axes (e.g., the A axis and B axis of FIG. 3). Referring to the unfolded state (500) of the multi-foldable electronic device (101), the multi-foldable electronic device (101) may include a first housing (510) (e.g., the first housing (315) of FIG. 3), a second housing (520) (e.g., the second housing (325) of FIG. 3), and a third housing (530) (e.g., the third housing (335) of FIG. 3). A multi-foldable electronic device (101) may include a first hinge structure (513) disposed between the first housing (510) and the second housing (520) so that the first housing (510) and the second housing (520) fold together, and a second hinge structure (515) disposed between the second housing (520) and the third housing (530) so that the second housing (520) and the third housing (530) fold together. A multi-foldable electronic device (101) may include a display (e.g., a display module (160) of FIG. 1) comprising a first display area (501) (e.g., the first display area (310) of FIG. 3) positioned corresponding to a first housing (510), a second display area (503) (e.g., the second display area (320) of FIG. 3) positioned between a first hinge structure (513) and a second hinge structure (515) corresponding to a second housing (520), and a third display area (505) (e.g., the third display area (330) of FIG. 3) positioned corresponding to a third housing (530). Hereinafter, FIG. 3 is used as an example, but the present invention is not limited by the description.

[0103] The multi-foldable electronic device (101) may be in a partially folded state in which either the first hinge structure (513) or the second hinge structure (515) is folded. The first partially folded state (550) is a state in which the first hinge structure (513) is folded, and only the third display area (505) may be displayable (e.g., usable). In the first partially folded state (550) of the multi-foldable electronic device (101), a first edge margin area (551) may occur to the left of the third display area (505) adjacent to the folded part of the multi-foldable electronic device (101) (e.g., the part where the first housing (510) and the second housing (520) are folded). The first edge margin area (551) may be a point (or location) where difficulty (e.g., difficulty in touching) may occur when a user touches the third display area (505). The first edge margin area (551) may vary depending on the type of input means. The input means may refer to a pen (e.g., a digitizer pen) or a user's finger that touches the third display area (505). The multi-foldable electronic device (101) can detect hover input by the pen when the user places the pen on the third display area (505) and moves a certain distance closer.

[0104] For example, if the thickness of the pen is thick, the first edge margin area (551) may be large, and if the thickness of the pen is thin, the first edge margin area (551) may be small. Alternatively, if the touch area of ​​the finger is small, the first edge margin area (551) may be small, and if the touch area of ​​the finger is large, the first edge margin area (551) may be large. The multi-foldable electronic device (101) can determine the window usage size of the third display area (505) by considering that the first edge margin area (551) varies depending on the type of input means. The window usage size may refer to the usable display area excluding the first edge margin area (551) from the third display area (505). For example, if the first edge margin area (551) becomes larger, the window usage size becomes smaller, and if the first edge margin area (551) becomes smaller, the window usage size may become larger. The first edge margin area (551) and the window usage size may be inversely proportional.

[0105] According to one embodiment, when the input means is a pen, the multi-foldable electronic device (101) can identify the thickness of the pen or the length of the pen tip corresponding to the pen from a memory (e.g., memory (130) of FIG. 1). The memory (130) may store the thickness of the pen or the length of the pen tip for each pen. When the pen is recognized (e.g., paired) by the input means, the multi-foldable electronic device (101) can identify the thickness of the recognized pen or the length of the pen tip from the memory (130). After performing pairing with the pen, the multi-foldable electronic device (101) can receive a sensing value in real time from the pen and calculate the orientation of the pen and / or the tilt of the pen based on the received sensing value. The multi-foldable electronic device (101) can calculate the direction of the pen and / or the tilt of the pen based on the sensing value received from the pen and the sensing information of the device (e.g., acceleration value, gyroscope sensor value) sensed by at least one sensor of the multi-foldable electronic device (101) (e.g., sensor module (176) of FIG. 1). The multi-foldable electronic device (101) can determine the window usage size of the third display area (505) based on at least one of the identified pen thickness, the identified pen tip length, the direction of the pen, or the tilt of the pen.

[0106] According to one embodiment, the multi-foldable electronic device (101) can analyze the touch area and shape of a finger touching a third display area (505) when the input means is a user's finger. The multi-foldable electronic device (101) can analyze the long axis of the touch, the short axis of the touch, the X,Y coordinate values ​​of the center point of the touch, or the tilt of the finger. The long axis of the touch may refer to the length of the longer axis relative to the center point of the touch by analyzing the shape of the touch. The short axis of the touch may refer to the length of the shorter axis relative to the center point of the touch by analyzing the shape of the touch. The multi-foldable electronic device (101) can determine the window usage size of the third display area based on the analysis results. The multi-foldable electronic device (101) can identify the type of finger in real time based on the touch area and shape, and change (or control, adjust) the layout of the screen based on the identified type of finger.

[0107] The second partial folding state (570) is a state in which the second hinge structure (515) is folded, and only the first display area (501) may be displayed. In the second partial folding state (570), the multi-foldable electronic device (101) may have a second edge margin area (571) to the right of the first display area (501) adjacent to the folded part of the multi-foldable electronic device (101) (e.g., the folded part of the second housing (520) and the third housing (530)). The second edge margin area (571) may be a point (or location) where difficulty (e.g., difficulty in touching) may occur when a user touches the first display area (501). The second edge margin area (571) may vary depending on the type of input means. The input means may refer to a pen or the user's finger that touches the first display area (501).

[0108] For example, if the thickness of the pen is thick, the second edge margin area (571) may be large, and if the thickness of the pen is thin, the second edge margin area (571) may be small. Alternatively, if the touch area of ​​the finger is small, the second edge margin area (571) may be small, and if the touch area of ​​the finger is large, the second edge margin area (571) may be large. The multi-foldable electronic device (101) can determine the window usage size of the first display area (501) by considering that the second edge margin area (571) varies depending on the type of input means. The window usage size may refer to the usable display area excluding the second edge margin area (571) from the first display area (501). For example, if the second edge margin area (571) becomes larger, the window usage size becomes smaller, and if the second edge margin area (571) becomes smaller, the window usage size may become larger. The second edge margin area (571) and the window usage size may be inversely proportional.

[0109] The multi-foldable electronic device (101) can change (or control, adjust) the layout of the screen displayed in the third display area (505) based on the determined window usage size. The screen displayed in the third display area (505) may, for example, mean a home screen, a lock screen, or an application execution screen. The multi-foldable electronic device (101) can control the screen displayed in the third display area (505) to correspond to the determined window usage size. According to one embodiment, if the multi-foldable electronic device (101) includes an object for user interaction (e.g., a first object) in the first edge margin area (551), the object for user interaction can be moved to an available display area of ​​the third display area (505). The object for user interaction may include any touchable object such as a software button, a link object capable of page navigation, and an icon. The multi-foldable electronic device (101) can significantly change the size of the object for the user interaction. The multi-foldable electronic device (101) can significantly change the touchable area of ​​the object for the user interaction. If the multi-foldable electronic device (101) includes an object that does not require user interaction (e.g., a second object) in the first edge margin area (551), the display of the second object can be maintained without moving the second object to the available display area of ​​the third display area (505).

[0110] A multi-foldable electronic device (101) according to one embodiment of the present disclosure comprises: a housing including a first housing (510), a second housing (520), and a third housing (530); at least one sensor (176) disposed in at least one of the first housing to the third housing; a first hinge structure (513) disposed between the first housing and the second housing so that the first housing and the second housing fold together; a second hinge structure (515) disposed between the second housing and the third housing so that the second housing and the third housing fold together; a first display area (501) disposed corresponding to the first housing; a second display area (503) disposed between the first hinge structure and the second hinge structure corresponding to the second housing; and a third display area (505) disposed corresponding to the third housing; a memory (130) for storing instructions; and The multi-foldable electronic device may include a processor (120), and when the instructions are executed individually and / or collectively by the processor, the multi-foldable electronic device may detect the folding state of the first hinge structure based on information obtained from the at least one sensor, identify the type of input means when the first hinge structure is in a folding state, determine the window usage size of the third display area based on the type of pen when the type of input means is a pen, determine the window usage size of the third display area based on the touch area and shape when the type of input means is a user's finger, and change the layout of the screen displayed in the third display area based on the determined window usage size.

[0111] When the above instructions are executed individually and / or collectively by the processor, the multi-foldable electronic device may identify the thickness of the pen or the length of the pen tip corresponding to the pen from the memory when the input means is a pen, calculate the orientation of the pen and / or the tilt of the pen, and determine the window usage size of the third display area based on at least one of the identified thickness of the pen, the identified length of the pen tip, the orientation of the pen, or the tilt of the pen.

[0112] When the above instructions are executed individually and / or collectively by the processor, the multi-foldable electronic device may receive a sensing value in real time from the pen after pairing with the pen, and calculate the direction of the pen and / or the tilt of the pen based on the received sensing value.

[0113] When the above instructions are executed individually and / or collectively by the processor, the multi-foldable electronic device may analyze the touch area and shape of a finger touching the third display area when the input means is a user's finger, and determine the window usage size of the third display area based on the touch area and shape.

[0114] When the above instructions are executed individually and / or collectively by the processor, the multi-foldable electronic device can identify the type of finger in real time based on the touch area and shape, and change the layout of the screen based on the identified type of finger.

[0115] When the above instructions are executed individually and / or collectively by the processor, the multi-foldable electronic device may detect the folding angle of the second hinge structure based on information obtained from the at least one sensor when the first hinge structure is in a folded state, and determine the window usage size of the third display area by further considering the folding angle of the second hinge structure.

[0116] When the above instructions are executed individually and / or collectively by the processor, the multi-foldable electronic device may detect a touch on the third display area while the layout of the changed screen is displayed in the third display area, and determine whether to process the touch based on the area of ​​the touch and the time of the touch.

[0117] When the above instructions are executed individually and / or collectively by the processor, the multi-foldable electronic device may determine whether the area of ​​the touch and the time of the touch correspond to a specified usage pattern, and if the area of ​​the touch and the time of the touch correspond to the specified usage pattern, the device may process the touch.

[0118] When the above instructions are executed individually and / or collectively by the processor, the multifoldable electronic device may determine whether a state change of the multifoldable electronic device is detected within a specified time when the area of ​​the touch and the time of the touch correspond to the specified usage pattern, and if a state change of the multifoldable electronic device is detected within the specified time, the touch may be processed.

[0119] When the above instructions are executed individually and / or collectively by the processor, the multi-foldable electronic device may determine whether the area of ​​the touch and the time of the touch correspond to a specified usage pattern, and if the area of ​​the touch and the time of the touch do not correspond to the specified usage pattern, the touch may be ignored.

[0120] FIG. 6 is a flowchart (600) illustrating the operation method of a multi-foldable electronic device according to one embodiment.

[0121] Referring to FIG. 6, in operation 601, a processor (e.g., processor (120) of FIG. 1) of a multi-foldable electronic device (e.g., multi-foldable electronic device (101) of FIG. 1) according to one embodiment can detect the folding state of a first hinge structure. The multi-foldable electronic device (101) can be folded, unfolded, or bent based on a first hinge structure (e.g., first hinge structure (513) of FIG. 5) or a second hinge structure (e.g., second hinge structure (515) of FIG. 5) corresponding to two folding axes (e.g., axis A and axis B of FIG. 5). A multi-foldable electronic device (101) may include a first housing (e.g., the first housing (510) of FIG. 5), a second housing (e.g., the second housing (520) of FIG. 5), and a third housing (e.g., the third housing (530) of FIG. 5). The foldable electronic device (101) may further include at least one sensor (e.g., the sensor module (176) of FIG. 1). For example, the foldable electronic device (101) may include a first sensor module in the first housing (510), a second sensor module in the second housing (520), and a third sensor module in the third housing (530). A processor (120) may detect the folding state of a first hinge structure (513) based on information obtained through the at least one sensor (e.g., the first sensor module, the second sensor module, or the third sensor module).

[0122] A foldable electronic device (101) has a first housing (510) and a second housing (520) that fold together, a first hinge structure (513) disposed between the first housing (510) and the second housing (520), a second hinge structure (515) disposed between the second housing (520) and the third housing (530) that fold together, a first display area disposed corresponding to the first housing (510) (e.g., the first display area (501) of FIG. 5), a second display area disposed corresponding to the second housing (520) and between the first hinge structure (513) and the second hinge structure (515) (e.g., the second display area (503) of FIG. 5), and a third display area disposed corresponding to the third housing (530) (e.g., the third display area of ​​FIG. 5). A display (e.g., the display module (160) of FIG. 1) including a region (505)) may be included. The folding state of the first hinge structure (513) may refer to the first partial folding state (550) of FIG. 5. Although the folding state of the first hinge structure (513) is described below, it can be implemented in the same way in the second partial folding state (570) of FIG. 5.

[0123] In operation 603, the processor (120) can identify the type of input means. The processor (120) can identify the type of input means in the folded state of the first hinge structure (513) (e.g., the first partially folded state (550) of FIG. 5). In the folded state of the first hinge structure (513), only the third display area (505) may be displayable (e.g., usable). The input means may refer to a pen or a user's finger that touches the third display area (505).

[0124] In operation 605, the processor (120) can determine whether the input means is a pen. The processor (120) may operate differently depending on whether the input means is a pen or a user's finger. The processor (120) may perform operation 607 if the input means is a pen, and perform operation 608 if the input means is a user's finger.

[0125] If the input means is a pen, in operation 607, the processor (120) can identify the type of the pen. The type of the pen may include the thickness of the pen or the length of the pen tip. The thickness of the pen or the length of the pen tip may be stored in the memory of the multi-foldable electronic device (101) (e.g., the memory (130) of FIG. 1). When the pen is recognized (e.g., paired) by the input means, the processor (120) can identify the thickness of the recognized pen or the length of the pen tip from the memory (130).

[0126] In operation 609, the processor (120) can determine the window usage size of the third display area (505) based on the type of the identified pen. When the first hinge structure (513) is in a folded state, an edge margin area (e.g., the first edge margin area (551) of FIG. 5) may occur on the left side of the third display area (505) adjacent to the folded part of the multi-foldable electronic device (101) (e.g., the folded part of the first housing (510) and the second housing (520)). The first edge margin area (551) may be a point (or location) where difficulty (e.g., poor touch) may occur when the user touches the third display area (505). Additionally, the first edge margin area (551) may be large when the pen is thick, and the first edge margin area (551) may be small when the pen is thin. Accordingly, the processor (120) can determine the window usage size of the third display area (505) based on the thickness of the pen and the length of the pen tip.

[0127] According to one embodiment, the processor (120) receives a sensing value in real time from the pen after performing pairing with the pen, and can calculate the direction of the pen and / or the tilt of the pen based on the received sensing value. According to an actual embodiment, the processor (120) can calculate the direction of the pen and / or the tilt of the pen based on the sensing value received from the pen and the sensing information of the device (e.g., acceleration value, gyroscope sensor value) sensed by at least one sensor of the multi-foldable electronic device (101) (e.g., sensor module (176) of FIG. 1). The closer the direction of the pen and / or the tilt of the pen is to the first edge margin area (551), the smaller the window size of the third display area (505) may be. The more the direction of the pen and / or the tilt of the pen is opposite to the first edge margin area (551), the larger the window size of the third display area (505) may be.

[0128] If the input means is a user's finger, in operation 608, the processor (120) can identify (or measure, detect) the touch area and shape of the finger. The display module (160) may include a touch sensor configured to detect a touch. The processor (120) can use the touch sensor to identify the touch area and shape whenever the user's finger comes into contact with the third display area (505).

[0129] In operation 610, the processor (120) can determine the window usage size of the third display area (505) based on the touch area and shape of the finger. If the touch area of ​​the finger is small, the first edge margin area (551) may be small, and if the touch area of ​​the finger is large, the first edge margin area (551) may be large. The processor (120) can analyze the touch area and shape of the finger touching the third display area (505). For example, the processor (120) can analyze the major axis of the touch, the minor axis of the touch, the X,Y coordinate values ​​of the center point of the touch, or the tilt of the finger. The processor (120) can determine the window usage size of the third display area (505) based on the analysis results.

[0130] In operation 611, the processor (120) can change (or control, adjust) the layout of the screen displayed in the third display area (505) based on the determined window usage size. The screen may refer to a home screen, a lock screen, or an application execution screen displayed in the third display area (505). The window usage size may refer to the available display area excluding the first edge margin area (551) in the third display area (505). For example, if the first edge margin area (551) increases, the window usage size decreases, and if the first edge margin area (551) decreases, the window usage size may increase. The first edge margin area (551) and the window usage size may have an inverse relationship. The processor (120) can control the screen displayed in the third display area (505) to correspond to the determined window usage size.

[0131] According to one embodiment, if the first edge margin area (551) contains an object for user interaction (e.g., a first object), the processor (120) may move the object for user interaction to an available display area of ​​the third display area (505). The object for user interaction may include any touchable object, such as a software button, a link object capable of page navigation, and an icon. The processor (120) may significantly change the size of the object for user interaction. The processor (120) may significantly change the touchable area of ​​the object for user interaction. If the first edge margin area (551) contains an object that does not require user interaction (e.g., a second object), the processor (120) may maintain the display of the second object without moving the second object to an available display area of ​​the third display area (505).

[0132] According to one embodiment, the processor (120) can identify the type of finger in real time based on the touch area and shape, and change the layout of the screen based on the identified type of finger.

[0133] According to one embodiment, the processor (120) can detect the folding angle of the second hinge structure (515) when the first hinge structure (513) is in a folding state, and determine the window usage size of the third display area (505) by further considering the folding angle of the second hinge structure (515). When the first hinge structure (513) is in a folding state, the second hinge structure (515) may be in an unfolded state or a folding state. The unfolded state of the second hinge structure (515) is a state in which the second housing (520) and the third housing (530) are arranged side by side, and the angle between the second housing (520) and the third housing (530) may be 180 degrees. The folding state of the second hinge structure (515) is a state in which the second housing (520) and the third housing (530) have a constant angle, and the angle between the second housing (520) and the third housing (530) may be 90 to 179 degrees.

[0134] The processor (120) can detect a touch on the third display area (505) while the layout of the changed screen is displayed on the third display area (505), and can determine whether to process the touch based on the area of ​​the touch and the time of the touch. For example, the processor (120) can determine whether the area of ​​the touch and the time of the touch correspond to a specified usage pattern, and if the area of ​​the touch and the time of the touch correspond to the specified usage pattern, it can process the touch. The specified usage pattern may mean that the area of ​​the touch detected during a specified time (e.g., 3 seconds, 4 seconds) corresponds to the specified area. If the area of ​​the touch and the time of the touch do not correspond to the specified usage pattern, the processor (120) can ignore the touch. For example, when a user folds the multi-foldable electronic device (101), they can touch the third display area (505) with the broad side of their thumb. In this case, the processor (120) may recognize a touch caused by thumb contact on the third display area (505) as a touch for folding and ignore the touch.

[0135] According to one embodiment, the processor (120) can determine whether a change in the state of the multi-foldable electronic device (101) is detected within a specified time if the area of ​​the touch and the time of the touch correspond to the specified usage pattern. A change in the state of the multi-foldable electronic device (101) may mean that the second hinge structure (515) changes from an unfolded state to a folded state. If a change in the state of the multi-foldable electronic device (101) is detected within the specified time, the processor (120) can process the touch. For example, if the user touches the multi-foldable electronic device (101) without folding it, the user can touch the third display area (505) with the side of the thumb held upright. In this case, the processor (120) can recognize the touch caused by the thumb contact on the third display area (505) as a touch intended by the user and process the touch.

[0136] FIG. 7a is a drawing illustrating an example in which a second hinge structure is in an unfolded state in a multi-foldable electronic device according to one embodiment.

[0137] Referring to FIG. 7a, a multi-foldable electronic device according to one embodiment (e.g., the multi-foldable electronic device (101) of FIG. 1) may be folded, unfolded, or bent based on a first hinge structure (e.g., the first hinge structure (513) of FIG. 5) or a second hinge structure (e.g., the second hinge structure (515) of FIG. 5) corresponding to two folding axes (e.g., the A axis and B axis of FIG. 5). Such a multi-foldable electronic device (101) may include a first housing (e.g., the first housing (510) of FIG. 5), a second housing (e.g., the second housing (520) of FIG. 5), and a third housing (e.g., the third housing (530) of FIG. 5). A multi-foldable electronic device (101) may include a first hinge structure (513) disposed between the first housing (510) and the second housing (520) so that the first housing (510) and the second housing (520) fold together, and a second hinge structure (515) disposed between the second housing (520) and the third housing (530) so that the second housing (520) and the third housing (530) fold together. A multi-foldable electronic device (101) may include a display (e.g., a display module (160) of FIG. 1) comprising a first display area (e.g., a first display area (501) of FIG. 5) positioned corresponding to a first housing (510), a second display area (e.g., a second display area (503) of FIG. 5) positioned between a first hinge structure (513) and a second hinge structure (515) corresponding to a second housing (520), and a third display area (e.g., a third display area (505) of FIG. 5) positioned corresponding to a third housing (530). The folding state of the first hinge structure (513) may refer to the first partial folding state (550) of FIG. 5. Although the following description describes the folding state of the first hinge structure (513), it can be implemented in the same way in the second partial folding state (570) of FIG. 5.

[0138] The first reference numeral (700) indicates the unfolded state of the second hinge structure (515) when the first hinge structure (513) is in a folded state. When the second hinge structure (515) is in an unfolded state, the angle between the second housing (520) and the third housing (530) may be 180 degrees. When the second hinge structure (515) is in an unfolded state, an edge margin area (e.g., the first edge margin area (551) of FIG. 5) may occur on the left side of the third display area (505) adjacent to the folded part of the first housing (510) and the second housing (520). The edge margin area may be a point (or location) where difficulty (e.g., difficulty in touching) may occur when a user touches the third display area (505).

[0139] FIG. 7b is a drawing illustrating an example in which a second hinge structure is in a folded state in a multi-foldable electronic device according to one embodiment.

[0140] The second reference numeral (750) indicates a state in which a portion of the second hinge structure (515) is folded while the first hinge structure (513) is in a folded state. When the second hinge structure (515) is partially folded, the angle between the second housing (520) and the third housing (530) may be 90 degrees to 179 degrees. Although the second reference numeral (750) is described using the example of the second hinge structure (515) having a folding angle of 90 degrees, the present invention is not limited by the drawings. When the second hinge structure (515) is in a folded state, an edge margin area (e.g., the first edge margin area (551) of FIG. 5) may occur to the left of the third display area (505) adjacent to the folded portion of the first housing (510) and the second housing (520). The edge margin area of ​​the first reference numeral (700) may be smaller than the edge margin area of ​​the second reference numeral (750). This is because a portion of the third display area (505) may be obscured by the folding of the second hinge structure (515).

[0141] Accordingly, the processor of the multi-foldable electronic device (101) (e.g., the processor (120) of FIG. 1) can determine an edge margin area based on whether the second hinge structure (515) is folded when the first hinge structure (513) is in a folded state. The processor (120) can determine the window usage size of the third display area (505) based on the type of input means (e.g., pen, finger) and the edge margin area. Here, if the edge margin area increases, the window usage size decreases, and if the edge margin area decreases, the window usage size may increase. The edge margin area and the window usage size may have an inverse relationship. The processor (120) can change the layout of the screen displayed in the third display area (505) based on the determined window usage size.

[0142] FIG. 8a is a drawing illustrating an example of guiding a designated screen mode when using a pen in a multi-foldable electronic device according to one embodiment.

[0143] Referring to FIG. 8a, a multi-foldable electronic device according to one embodiment (e.g., the multi-foldable electronic device (101) of FIG. 1) may be folded, unfolded, or bent based on a first hinge structure (e.g., the first hinge structure (513) of FIG. 5) or a second hinge structure (e.g., the second hinge structure (515) of FIG. 5) corresponding to two folding axes (e.g., the A axis and B axis of FIG. 5). Such a multi-foldable electronic device (101) may include a first housing (e.g., the first housing (510) of FIG. 5), a second housing (e.g., the second housing (520) of FIG. 5), and a third housing (e.g., the third housing (530) of FIG. 5). A multi-foldable electronic device (101) may include a first hinge structure (513) disposed between the first housing (510) and the second housing (520) so that the first housing (510) and the second housing (520) fold together, and a second hinge structure (515) disposed between the second housing (520) and the third housing (530) so that the second housing (520) and the third housing (530) fold together. A multi-foldable electronic device (101) may include a display (e.g., a display module (160) of FIG. 1) comprising a first display area (e.g., a first display area (501) of FIG. 5) positioned corresponding to a first housing (510), a second display area (e.g., a second display area (503) of FIG. 5) positioned between a first hinge structure (513) and a second hinge structure (515) corresponding to a second housing (520), and a third display area (e.g., a third display area (505) of FIG. 5) positioned corresponding to a third housing (530). The folding state of the first hinge structure (513) may refer to the first partial folding state (550) of FIG. 5. Although the following description describes the folding state of the first hinge structure (513), it can be implemented in the same way in the second partial folding state (570) of FIG. 5.

[0144] A processor of a multi-foldable electronic device (101) (e.g., processor (120) of FIG. 1) can display a first user interface (810) on a third display area (505) of a display module (160) when the first hinge structure (513) is in a folded state (800). The processor (120) can recognize that the input means is a pen (801) (e.g., a digitizer pen) and can enter a designated screen mode (e.g., a fitting mode). The processor (120) can detect a hover input by the pen when the user places the pen on the display module (160) and moves a certain distance closer. The first user interface (810) may guide the user to the designated screen mode when using the pen (801).

[0145] FIG. 8b is a drawing illustrating an example of calculating the direction and tilt of a pen in a multi-foldable electronic device according to one embodiment.

[0146] Referring to FIG. 8b, the processor (120) can calculate the direction of the pen (850) and / or the tilt of the pen (870) after entering a designated screen mode. The direction of the pen (850) may refer to the orientation of the display module (160) and the pen (801). The direction of the pen (850) may refer to a direction of 0 degrees to ±180 degrees (or 0 degrees to 360 degrees). The first direction of the pen (851) may indicate that the pen (801) is positioned at a 90-degree direction from the display module (160). The second direction of the pen (853) may indicate that the pen (801) is positioned at a -45-degree direction from the display module (160). The closer the direction of the pen (850) is to the folded part of the multi-foldable electronic device (101) (e.g., the folded part of the first housing (510) and the second housing (520)), the smaller the window usage size can be.

[0147] The pen tilt (870) may refer to the degree to which the pen (801) is tilted from the display module (160). The tilt angle (871) of the pen (801) can be calculated based on a 90-degree direction from the display module (160). The closer the pen tilt (870) is to the folded part of the multi-foldable electronic device (101) (e.g., the folded part of the first housing (510) and the second housing (520)), the smaller the window usage size may be.

[0148] According to one embodiment, a processor of a multi-foldable electronic device (101) (e.g., processor (120) of FIG. 1) may receive a sensing value in real time from a pen (801) and calculate the direction of the pen and / or the tilt of the pen based on the received sensing value. The processor (120) may calculate the direction of the pen and / or the tilt of the pen based on the sensing value received from the pen and the sensing information of the device (e.g., acceleration value, gyroscope sensor value) sensed by at least one sensor of the multi-foldable electronic device (101) (e.g., sensor module (176) of FIG. 1). The thickness of the pen and the length of the pen tip may be stored in the memory of the multi-foldable electronic device (101) (e.g., memory (130) of FIG. 1). The processor (120) may identify the thickness of the pen and the length of the pen tip corresponding to the pen (801) in the memory (130). The processor (120) can determine the window usage size of the third display area (505) of the display module (160) based on at least one of the thickness of the identified pen, the length of the identified pen tip, the direction of the pen, or the inclination of the pen.

[0149] FIG. 9 is a flowchart (900) illustrating a method for determining the window usage size according to the type of pen of a multi-foldable electronic device according to one embodiment. FIG. 9 may embody operations 603 to 609 of FIG. 6.

[0150] Referring to FIG. 9, in operation 901, a processor (e.g., processor (120) of FIG. 1) of a multi-foldable electronic device (e.g., multi-foldable electronic device (101) of FIG. 1) according to one embodiment can detect a folding state of a second hinge structure. The multi-foldable electronic device (101) can be folded, unfolded, or bent based on a first hinge structure (e.g., first hinge structure (513) of FIG. 5) or a second hinge structure (e.g., second hinge structure (515) of FIG. 5) corresponding to two folding axes (e.g., axis A and axis B of FIG. 5). A multi-foldable electronic device (101) may include a first housing (e.g., the first housing (510) of FIG. 5), a second housing (e.g., the second housing (520) of FIG. 5), and a third housing (e.g., the third housing (530) of FIG. 5). The foldable electronic device (101) may further include at least one sensor (e.g., the sensor module (176) of FIG. 1). For example, the foldable electronic device (101) may include a first sensor module in the first housing (510), a second sensor module in the second housing (520), and a third sensor module in the third housing (530). A processor (120) may detect the folding state of a second hinge structure (515) based on information obtained through the at least one sensor (e.g., the first sensor module, the second sensor module, or the third sensor module).

[0151] A multi-foldable electronic device (101) may include a first hinge structure (513) disposed between the first housing (510) and the second housing (520) so that the first housing (510) and the second housing (520) fold together, and a second hinge structure (515) disposed between the second housing (520) and the third housing (530) so that the second housing (520) and the third housing (530) fold together. A multi-foldable electronic device (101) may include a display (e.g., a display module (160) of FIG. 1) comprising a first display area (e.g., a first display area (501) of FIG. 5) positioned corresponding to a first housing (510), a second display area (e.g., a second display area (503) of FIG. 5) positioned between a first hinge structure (513) and a second hinge structure (515) corresponding to a second housing (520), and a third display area (e.g., a third display area (505) of FIG. 5) positioned corresponding to a third housing (530). The folding state of the first hinge structure (513) may refer to the first partial folding state (550) of FIG. 5. Although the following description describes the folding state of the first hinge structure (513), it can be implemented in the same way in the second partial folding state (570) of FIG. 5. The processor (120) can detect the folding state of the second hinge structure (515) when the first hinge structure (513) is in a folding state (e.g., after operation 601).

[0152] In operation 903, the processor (120) can determine whether the second hinge structure (515) is in a folding state. If the second hinge structure (515) is in a folding state, the processor (120) performs operation 905, and if the second hinge structure (515) is not in a folding state (e.g., unfolded state), it can perform operation 906.

[0153] When the second hinge structure (515) is in a folded state, in operation 905, the processor (120) can determine an edge margin area based on the folding angle. The processor (120) can detect the folding angle of the second hinge structure (515) to determine how much the second hinge structure (515) is folded. The larger the folding angle, the smaller the edge margin area may be. This is because a portion of the third display area (505) may be obscured by the folding of the second hinge structure (515). If the folding angle is large (e.g., 90 degrees), the third display area (505) may be more obscured by the folded portion of the first housing (510) and the second housing (520). If the folding angle is small (e.g., 30 degrees), the third display area (505) may be covered less by the folded portion of the first housing (510) and the second housing (520).

[0154] When the second hinge structure (515) is not in a folded state (e.g., unfolded state), in operation 906, the processor (120) can determine a designated edge margin area. When the second hinge structure (515) is in an unfolded state, the processor (120) can determine the edge margin area as a designated edge margin area. When the second hinge structure (515) is in an unfolded state, a difficult-to-touch area in the third display area (505) may be determined by the folded portion of the first housing (510) and the second housing (520). The designated edge margin area may mean that the edge margin area has already been determined according to an algorithm. After determining the edge margin area, the processor (120) can perform operation 907.

[0155] In operation 907, the processor (120) can identify the thickness of the pen and the length of the pen tip. The thickness of the pen and the length of the pen tip for each pen may be stored in the memory of the multi-foldable electronic device (101) (e.g., the memory (130) of FIG. 1). When the processor (120) detects a pen (e.g., the pen (801) of FIG. 8a), it can pair with the pen (801). The processor (120) can identify the thickness of the pen and the length of the pen tip corresponding to the paired pen (801) in the memory (130).

[0156] In operation 909, the processor (120) can calculate the direction of the pen and the tilt of the pen. The processor (120) receives a sensing value from the paired pen (801) and can calculate the direction of the pen and the tilt of the pen based on the received sensing value. Although operations 907 and 909 are depicted as separate operations in the drawings to aid in understanding the invention, operations 907 and 909 may be performed simultaneously or as a single operation. Operation 909 may be performed first and operation 907 may be performed later.

[0157] In operation 911, the processor (120) may determine the window usage size based on the edge margin area and the input information of the pen. The input information of the pen may include at least one of the thickness of the pen, the length of the pen tip, the direction of the pen, and / or the tilt of the pen. The thicker the pen, the smaller the window usage size may be. The shorter the length of the pen tip, the smaller the window usage size may be. The direction of the pen may be closer to the folded part of the multi-foldable electronic device (101) (e.g., the folded part of the first housing (510) and the second housing (520)), the smaller the window usage size may be. The tilt of the pen may be closer to the folded part of the multi-foldable electronic device (101) (e.g., the folded part of the first housing (510) and the second housing (520)), the smaller the window usage size may be.

[0158] When the window usage size is determined, the processor (120) can perform operation 611 of FIG. 6 to change (or control, adjust) the layout of the screen displayed in the third display area (505) based on the window usage size.

[0159] According to one embodiment, if an object for user interaction (e.g., a first object) is included in the edge margin area, the processor (120) may move the object for user interaction to an available display area of ​​the third display area (505). The object for user interaction may include any touchable object such as a software button, a link object capable of page navigation, and an icon. The processor (120) may significantly change the size of the object for user interaction. The processor (120) may significantly change the touchable area of ​​the object for user interaction. If an object that does not require user interaction (e.g., a second object) is included in the first edge margin area (551), the processor (120) may maintain the display of the second object without moving the second object to an available display area of ​​the third display area (505).

[0160] FIG. 10a is a drawing illustrating an example of determining the size of a first window according to the direction and tilt of a pen in a multi-foldable electronic device according to one embodiment.

[0161] Referring to FIG. 10a, a multi-foldable electronic device according to one embodiment (e.g., the multi-foldable electronic device (101) of FIG. 1) may be folded, unfolded, or bent based on a first hinge structure (e.g., the first hinge structure (513) of FIG. 5) or a second hinge structure (e.g., the second hinge structure (515) of FIG. 5) corresponding to two folding axes (e.g., the A axis and B axis of FIG. 5). Such a multi-foldable electronic device (101) may include a first housing (e.g., the first housing (510) of FIG. 5), a second housing (e.g., the second housing (520) of FIG. 5), and a third housing (e.g., the third housing (530) of FIG. 5). A multi-foldable electronic device (101) may include a first hinge structure (513) disposed between the first housing (510) and the second housing (520) so that the first housing (510) and the second housing (520) fold together, and a second hinge structure (515) disposed between the second housing (520) and the third housing (530) so that the second housing (520) and the third housing (530) fold together. A multi-foldable electronic device (101) may include a display (e.g., a display module (160) of FIG. 1) comprising a first display area (e.g., a first display area (501) of FIG. 5) positioned corresponding to a first housing (510), a second display area (e.g., a second display area (503) of FIG. 5) positioned between a first hinge structure (513) and a second hinge structure (515) corresponding to a second housing (520), and a third display area (e.g., a third display area (505) of FIG. 5) positioned corresponding to a third housing (530). The folding state of the first hinge structure (513) may refer to the first partial folding state (550) of FIG. 5. Although the following description describes the folding state of the first hinge structure (513), it can be implemented in the same way in the second partial folding state (570) of FIG. 5.

[0162] Referring to the first pen state (1000), the processor of the multi-foldable electronic device (101) (e.g., the processor (120) of FIG. 1) can calculate the direction of the pen (801) and / or the tilt of the pen (801) when the first hinge structure (513) is in a folded state. The processor (120) can determine whether the direction of the pen (801) and / or the tilt of the pen (801) is adjacent to a folded part of the multi-foldable electronic device (101) (e.g., a folded part of the first housing (510) and the second housing (520)). In the first pen state (1000), the processor (120) can determine the first window usage size (1010) based on the direction of the pen (801) and / or the tilt of the pen (801).

[0163] FIG. 10b is a drawing illustrating an example of determining the size of a second window according to the direction and tilt of a pen in a multi-foldable electronic device according to one embodiment.

[0164] Referring to FIG. 10b, the processor (120) can calculate the direction of the pen (801) and / or the tilt of the pen (801) when the first hinge structure (513) is in a folded state. Referring to the second pen state (1050), the processor (120) can determine whether the direction of the pen (801) and / or the tilt of the pen (801) is adjacent to a folded part of the multi-foldable electronic device (101) (e.g., a folded part of the first housing (510) and the second housing (520)). In the second pen state (1050), the processor (120) can determine the second window usage size (1020) based on the direction of the pen (801) and / or the tilt of the pen (801). The second window usage size (1020) may be larger than the first window usage size (1010). This may be because, in the second pen state (1050), the direction and / or tilt of the pen (801) is not adjacent to the folded part of the foldable electronic device (101) as in the first pen state (1000), the direction and / or tilt of the pen (801) is.

[0165] FIG. 10c is a drawing illustrating an example of determining the size of a third window according to the direction and tilt of a pen in a multi-foldable electronic device according to one embodiment.

[0166] Referring to FIG. 10c, the processor (120) can calculate the direction of the pen (801) and / or the tilt of the pen (801) when the first hinge structure (513) is in a folded state. Referring to the third pen state (1070), the processor (120) can determine whether the direction of the pen (801) and / or the tilt of the pen (801) is adjacent to a folded part of the multi-foldable electronic device (101) (e.g., a folded part of the first housing (510) and the second housing (520)). In the third pen state (1070), the processor (120) can determine the third window usage size (1030) based on the direction of the pen (801) and / or the tilt of the pen (801). The third window usage size (1030) may be smaller than the first window usage size (1010) or the second window usage size (1020). This may be because, in the third pen state (1070), the direction and / or tilt of the pen (801) is located closer to the folded part of the foldable electronic device (101) than the direction and / or tilt of the pen (801) in the first pen state (1000) or the second pen state (1050).

[0167] FIG. 11a is a drawing illustrating an example of guiding a designated screen mode when a user uses their finger in a multi-foldable electronic device according to one embodiment.

[0168] Referring to FIG. 11a, a multi-foldable electronic device according to one embodiment (e.g., multi-foldable electronic device (101) of FIG. 1) may be folded, unfolded, or bent based on a first hinge structure (e.g., first hinge structure (513) of FIG. 5) or a second hinge structure (e.g., second hinge structure (515) of FIG. 5) corresponding to two folding axes (e.g., axis A and axis B of FIG. 5). Such a multi-foldable electronic device (101) comprises a first housing (e.g., the first housing (510) of FIG. 5), a second housing (e.g., the second housing (520) of FIG. 5), and a third housing (e.g., the third housing (530) of FIG. 5), a first hinge structure (513) disposed between the first housing (510) and the second housing (520) so that the first housing (510) and the second housing (520) fold together, a second hinge structure (515) disposed between the second housing (520) and the third housing (530) so that the second housing (520) and the third housing (530) fold together, a first display area (e.g., the first display area (501) of FIG. 5) disposed corresponding to the first housing (510), and a first hinge structure (513) and a second hinge It may include a display (e.g., a display module (160) of FIG. 1) comprising a second display area (e.g., a second display area (503) of FIG. 5) disposed between the structures (515) and a third display area (e.g., a third display area (505) of FIG. 5) disposed corresponding to the third housing (530). The folding state of the first hinge structure (513) may refer to the first partial folding state (550) of FIG. 5. Although the following description describes the folding state of the first hinge structure (513), it can be implemented in the same way in the second partial folding state (570) of FIG. 5.

[0169] When the multi-foldable electronic device (101) is in a folded state (1100) of the first hinge structure (513), it can display a first user interface (1110) on a third display area (505) of the display module (160). The multi-foldable electronic device (101) can recognize that the input means is the user's finger (1101) and enter a designated screen mode (e.g., fitting mode). The first user interface (1110) may guide the user to the designated screen mode when the user's finger (1101) is used as the input means.

[0170] FIG. 11b is a drawing illustrating an example of calculating the touch area and shape of a finger in a multi-foldable electronic device according to one embodiment.

[0171] Referring to FIG. 11b, the processor (120) can calculate the touch area and shape of the touch when it detects a touch (e.g., a single touch) of the user's finger (1101) after entering a designated screen mode. The touch area and shape may include at least one of the major axis of the touch, the minor axis of the touch, the X,Y coordinate values ​​of the center point of the touch, or the inclination of the finger. Referring to the reference numeral (1130), the major axis of the touch (1131) may refer to the length of the longer axis relative to the center point of the touch by analyzing the shape of the touch. The minor axis of the touch (1133) may refer to the length of the shorter axis relative to the center point of the touch by analyzing the shape of the touch. The processor (120) can analyze the X,Y coordinate values ​​of the center point of the touch or the inclination of the finger (1135) based on the major axis of the touch (1131) and the minor axis of the touch (1133). The processor (120) can determine the window usage size of the third display area based on the analysis results. The processor (120) can identify the type of finger in real time based on the touch area and shape, and change (or control, adjust) the layout of the screen based on the identified type of finger.

[0172] FIG. 12 is a flowchart (1200) illustrating a method for determining the window usage size according to the touch area and shape of a finger of a multi-foldable electronic device according to one embodiment. FIG. 9 may be an embodiment of operations 603 to 609 of FIG. 6.

[0173] Referring to FIG. 12, in operation 1201, a processor (e.g., processor (120) of FIG. 1) of a multi-foldable electronic device (e.g., multi-foldable electronic device (101) of FIG. 1) according to one embodiment can detect a folding state of a second hinge structure. The multi-foldable electronic device (101) can be folded, unfolded, or bent based on a first hinge structure (e.g., first hinge structure (513) of FIG. 5) or a second hinge structure (e.g., second hinge structure (515) of FIG. 5) corresponding to two folding axes (e.g., axis A and axis B of FIG. 5). A multi-foldable electronic device (101) may include a first housing (e.g., the first housing (510) of FIG. 5), a second housing (e.g., the second housing (520) of FIG. 5), and a third housing (e.g., the third housing (530) of FIG. 5). The foldable electronic device (101) may further include at least one sensor (e.g., the sensor module (176) of FIG. 1). For example, the foldable electronic device (101) may include a first sensor module in the first housing (510), a second sensor module in the second housing (520), and a third sensor module in the third housing (530). A processor (120) may detect the folding state of a second hinge structure (515) based on information obtained through the at least one sensor (e.g., the first sensor module, the second sensor module, or the third sensor module).

[0174] A multi-foldable electronic device (101) has a first housing (510) and a second housing (520) that fold together, a first hinge structure (513) disposed between the first housing (510) and the second housing (520), a second hinge structure (515) disposed between the second housing (520) and the third housing (530) that fold together, a first display area disposed corresponding to the first housing (510) (e.g., the first display area (501) of FIG. 5), a second display area disposed corresponding to the second housing (520) and between the first hinge structure (513) and the second hinge structure (515) (e.g., the second display area (503) of FIG. 5), and a third display area disposed corresponding to the third housing (530) (e.g., the third display area of ​​FIG. 5). A display (e.g., the display module (160) of FIG. 1) including a region (505)) may be included. The folding state of the first hinge structure (513) may refer to the first partial folding state (550) of FIG. 5. Although the folding state of the first hinge structure (513) is described below, it can be implemented in the same way in the second partial folding state (570) of FIG. 5. The processor (120) can detect the folding state of the second hinge structure (515) when the first hinge structure (513) is in a folding state (e.g., after operation 601).

[0175] In operation 1203, the processor (120) can determine whether the second hinge structure (515) is in a folded state. If the second hinge structure (515) is in a folded state, the processor (120) performs operation 1205, and if the second hinge structure (515) is not in a folded state (e.g., unfolded state), it can perform operation 1206.

[0176] When the second hinge structure (515) is in a folded state, in operation 1205, the processor (120) can determine an edge margin area based on the folding angle. The processor (120) can detect the folding angle of the second hinge structure (515) to determine how much the second hinge structure (515) is folded. The larger the folding angle, the smaller the edge margin area may be. This is because a portion of the third display area (505) may be obscured by the folding of the second hinge structure (515). If the folding angle is large (e.g., 90 degrees), more of the third display area (505) may be obscured by the folded portion of the first housing (510) and the second housing (520). If the folding angle is small (e.g., 30 degrees), the third display area (505) may be covered less by the folded portion of the first housing (510) and the second housing (520).

[0177] When the second hinge structure (515) is not in a folded state (e.g., unfolded state), in operation 1206, the processor (120) can determine a designated edge margin area. When the second hinge structure (515) is in an unfolded state, the processor (120) can determine the edge margin area as a designated edge margin area. When the second hinge structure (515) is in an unfolded state, a difficult-to-touch area in the third display area (505) may be determined by the folded portion of the first housing (510) and the second housing (520). The designated edge margin area may mean that the edge margin area has already been determined according to an algorithm. After determining the edge margin area, the processor (120) can perform operation 1207.

[0178] In operation 1207, the processor (120) can identify the touch area of ​​a finger (e.g., finger (1101) of FIG. 11). The processor (120) can identify the touch area by analyzing the long axis or the short axis of the touch (e.g., single touch) of the finger (1110) detected by the display module (160). The long axis of the touch may refer to the length of the longer axis relative to the center point of the touch by analyzing the shape of the touch. The short axis of the touch may refer to the length of the shorter axis relative to the center point of the touch by analyzing the shape of the touch.

[0179] In operation 1209, the processor (120) can calculate (or analyze) the touch shape of the finger. The processor (120) can analyze the touch shape based on the X,Y coordinate values ​​of the center point of the touch or the tilt of the finger. Although operations 1207 and 1209 are depicted as separate operations in the drawings to aid in understanding the invention, operations 1207 and 1209 may be performed simultaneously or as a single operation. Operation 1209 may be performed first and operation 1207 may be performed later.

[0180] In operation 1211, the processor (120) can determine the window usage size based on the edge margin area and the input information of the hand. The input information of the hand may include at least one of the touch area and shape, for example, the major axis of the touch, the minor axis of the touch, the X,Y coordinate values ​​of the center point of the touch, and / or the tilt of the finger. The larger the touch area, the smaller the window usage size may be. The processor (120) can identify the type of finger that touched the display module (160) based on the touch area and the touch shape. For example, the processor (120) can identify it as a thumb if the touch area is large and the touch shape is a specified shape (e.g., an elongated shape). The processor (120) can identify it as a second finger based on the touch area and the touch shape.

[0181] When the window usage size is determined, the processor (120) can perform operation 611 of FIG. 6 to change (or control, adjust) the layout of the screen displayed in the third display area (505) based on the window usage size.

[0182] According to one embodiment, if an object for user interaction (e.g., a first object) is included in the edge margin area, the processor (120) may move the object for user interaction to an available display area of ​​the third display area (505). The object for user interaction may include any touchable object such as a software button, a link object capable of page navigation, and an icon. The processor (120) may significantly change the size of the object for user interaction. The processor (120) may significantly change the touchable area of ​​the object for user interaction. If an object that does not require user interaction (e.g., a second object) is included in the first edge margin area (551), the processor (120) may maintain the display of the second object without moving the second object to an available display area of ​​the third display area (505).

[0183] FIG. 13a is a drawing illustrating an example of determining the size of a first window used according to the touch area and shape of a finger in a multi-foldable electronic device according to one embodiment.

[0184] Referring to FIG. 13a, a multi-foldable electronic device according to one embodiment (e.g., multi-foldable electronic device (101) of FIG. 1) may be folded, unfolded, or bent based on a first hinge structure (e.g., first hinge structure (513) of FIG. 5) or a second hinge structure (e.g., second hinge structure (515) of FIG. 5) corresponding to two folding axes (e.g., axis A and axis B of FIG. 5). Such a multi-foldable electronic device (101) comprises a first housing (e.g., the first housing (510) of FIG. 5), a second housing (e.g., the second housing (520) of FIG. 5), and a third housing (e.g., the third housing (530) of FIG. 5), a first hinge structure (513) disposed between the first housing (510) and the second housing (520) so that the first housing (510) and the second housing (520) fold together, a second hinge structure (515) disposed between the second housing (520) and the third housing (530) so that the second housing (520) and the third housing (530) fold together, a first display area (e.g., the first display area (501) of FIG. 5) disposed corresponding to the first housing (510), and a first hinge structure (513) and a second hinge It may include a display (e.g., a display module (160) of FIG. 1) comprising a second display area (e.g., a second display area (503) of FIG. 5) disposed between the structures (515) and a third display area (e.g., a third display area (505) of FIG. 5) disposed corresponding to the third housing (530). The folding state of the first hinge structure (513) may refer to the first partial folding state (550) of FIG. 5. Although the following description describes the folding state of the first hinge structure (513), it can be implemented in the same way in the second partial folding state (570) of FIG. 5.

[0185] Referring to the first touch state (1300), the processor of the multi-foldable electronic device (101) (e.g., the processor (120) of FIG. 1) can calculate (or analyze) the touch area and shape of the finger (1101) when the first hinge structure (513) is in a folded state. The processor (120) can determine the window usage size to be smaller as the touch area is larger, and the window usage size to be larger as the touch area is smaller. In the first touch state (1300), the processor (120) can determine the first window usage size (1310) based on the touch area and shape of the finger (1101).

[0186] FIG. 13b is a drawing illustrating an example of processing a touch according to the touch area and touch time of a finger in a multi-foldable electronic device according to one embodiment.

[0187] Referring to FIG. 13b, the processor (120) can determine whether the area of ​​the touch (1301) of the finger (1101) detected through the third display area (505) and the time of the touch (1301) correspond to a specified usage pattern when the first hinge structure (513) is in a folded state. The specified usage pattern may mean processing the touch by analyzing the pattern in which the user uses the multi-foldable electronic device (101). The second touch state (1330) may be when the area of ​​the touch (1301) and the time of the touch (1301) correspond to the specified usage pattern. The processor (120) can process the touch (1301) when the area of ​​the touch (1301) and the time of the touch (1301) correspond to the specified usage pattern. The meaning of processing a touch (1301) may be to process the drawing if the touch (1301) is a drawing, or to perform a function corresponding to the button touched (1301) if the touch (1301) is selecting a button.

[0188] FIG. 14a is a drawing illustrating an example of processing a touch according to the touch area and touch time of a finger in a multi-foldable electronic device according to one embodiment.

[0189] Referring to FIG. 14a, a multi-foldable electronic device according to one embodiment (e.g., multi-foldable electronic device (101) of FIG. 1) may be folded, unfolded, or bent based on a first hinge structure (e.g., first hinge structure (513) of FIG. 5) or a second hinge structure (e.g., second hinge structure (515) of FIG. 5) corresponding to two folding axes (e.g., axis A and axis B of FIG. 5). Such a multi-foldable electronic device (101) comprises a first housing (e.g., the first housing (510) of FIG. 5), a second housing (e.g., the second housing (520) of FIG. 5), and a third housing (e.g., the third housing (530) of FIG. 5), a first hinge structure (513) disposed between the first housing (510) and the second housing (520) so that the first housing (510) and the second housing (520) fold together, a second hinge structure (515) disposed between the second housing (520) and the third housing (530) so that the second housing (520) and the third housing (530) fold together, a first display area (e.g., the first display area (501) of FIG. 5) disposed corresponding to the first housing (510), and a first hinge structure (513) and a second hinge It may include a display (e.g., a display module (160) of FIG. 1) comprising a second display area (e.g., a second display area (503) of FIG. 5) disposed between the structures (515) and a third display area (e.g., a third display area (505) of FIG. 5) disposed corresponding to the third housing (530). The folding state of the first hinge structure (513) may refer to the first partial folding state (550) of FIG. 5. Although the following description describes the folding state of the first hinge structure (513), it can be implemented in the same way in the second partial folding state (570) of FIG. 5.

[0190] The first state (1400) may indicate that the processor (120) processes the touch (1401) based on the area of ​​the touch (1401) and the time of the touch (1401). The processor (120) may determine whether the area of ​​the touch (1401) and the time of the touch (1401) correspond to the specified usage pattern, and if the area of ​​the touch (1401) and the time of the touch (1401) correspond to the specified usage pattern, it may determine whether a state change of the multi-foldable electronic device (101) is detected within a specified time. If a state change of the multi-foldable electronic device (101) is detected within the specified time, the processor (120) may process the touch (1401). A state change of the foldable electronic device (101) may mean that the first hinge structure (513) is unfolded or the second hinge structure (515) is folded.

[0191] FIG. 14b is a diagram illustrating an example of detecting a touch in a multi-foldable electronic device according to one embodiment and ignoring processing due to a state change.

[0192] Referring to FIG. 14b, the processor (120) may ignore (e.g., not process) the touch (1403) if the area of ​​the touch (1403) and the time of the touch (1403) do not correspond to the specified usage pattern. The second state (1450) is such that the processor (120) may determine whether a state change of the multi-foldable electronic device (101) is detected within a specified time if the area of ​​the touch (1403) and the time of the touch (1403) do not correspond to the specified usage pattern. The processor (120) may ignore the touch (1403) if a state change of the multi-foldable electronic device (101) is detected within the specified time. A state change of the foldable electronic device (101) may mean that the first hinge structure (513) is unfolded or the second hinge structure (515) is folded. For example, when a user folds the multi-foldable electronic device (101), they may touch the third display area (505) with the broad side of their thumb. Additionally, when a user folds the multi-foldable electronic device (101), no movement may be detected while the third display area (505) is being touched. The processor (120) may recognize the touch caused by the thumb contact on the third display area (505) as a touch for folding and ignore the touch.

[0193] A method of operation of a multi-foldable electronic device (101) comprising a housing including a first housing (510), a second housing (520), and a third housing (530) according to one embodiment of the present disclosure, at least one sensor (176) disposed in at least one of the first housing to the third housing, a first hinge structure (513) disposed between the first housing and the second housing so that the first housing and the second housing fold together, a second hinge structure (515) disposed between the second housing and the third housing so that the second housing and the third housing fold together, a first display area (501) disposed corresponding to the first housing, a second display area (503) disposed between the first hinge structure and the second hinge structure corresponding to the second housing, and a third display area (505) disposed corresponding to the third housing, wherein the first hinge The method may include an operation to detect the folding state of the structure, an operation to identify the type of input means when the first hinge structure is in a folding state, an operation to determine the window usage size of the third display area based on the type of pen when the type of input means is a pen, an operation to determine the window usage size of the third display area based on the touch area and shape when the type of input means is a user's finger, and an operation to change the layout of the screen displayed in the third display area based on the determined window usage size.

[0194] The above-determining operation may include, when the input means is a pen, identifying the thickness of the pen or the length of the pen tip corresponding to the pen from the memory, calculating the direction of the pen and / or the inclination of the pen, and determining the window usage size of the third display area based on at least one of the identified thickness of the pen, the identified length of the pen tip, the direction of the pen, or the inclination of the pen.

[0195] The above-described calculating operation may include the operation of receiving a sensing value in real time from the pen after pairing with the pen, and the operation of calculating the direction of the pen and / or the tilt of the pen based on the received sensing value.

[0196] The above-determining operation may include, when the input means is a user's finger, an operation of analyzing the touch area and shape of a finger touching the third display area, and an operation of determining the window usage size of the third display area based on the touch area and shape.

[0197] The above method may include an operation of identifying the type of finger in real time based on the touch area and shape, and an operation of changing the layout of the screen based on the identified type of finger.

[0198] The above-determining operation may include an operation of detecting the folding angle of the second hinge structure based on information obtained from the at least one sensor when the first hinge structure is in a folded state, and an operation of determining the window usage size of the third display area by further considering the folding angle of the second hinge structure.

[0199] The above method may further include an operation of detecting a touch on the third display area while the layout of the changed screen is displayed in the third display area, and an operation of determining whether to process the touch based on the area of ​​the touch and the time of the touch.

[0200] The operation of determining whether to process the touch may include an operation of determining whether the area of ​​the touch and the time of the touch correspond to a specified usage pattern, and an operation of processing the touch when the area of ​​the touch and the time of the touch correspond to the specified usage pattern.

[0201] The operation of determining whether to process the touch may include, when the area of ​​the touch and the time of the touch correspond to the specified usage pattern, determining whether a state change of the multi-foldable electronic device is detected within a specified time, and when a state change of the multi-foldable electronic device is detected within the specified time, processing the touch.

[0202] The operation of determining whether to process the touch may include an operation of determining whether the area of ​​the touch and the time of the touch correspond to a specified usage pattern, and an operation of ignoring the touch if the area of ​​the touch and the time of the touch do not correspond to the specified usage pattern.

[0203] The electronic device according to the various embodiments disclosed in this document may be of various forms. The electronic device may include, for example, a portable communication device (e.g., a smartphone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a consumer electronics device. The electronic device according to the embodiments of this document is not limited to the devices described above.

[0204] The various embodiments of this document and the terms used therein are not intended to limit the technical features described in this document to specific embodiments, and should be understood to include various modifications, equivalents, or substitutions of said embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of said items unless the relevant context clearly indicates otherwise. In this document, phrases such as "A or B," "at least one of A and B," "at least one of A or B," "A, B or C," "at least one of A, B and C," and "at least one of A, B, or C" may each include any one of the items listed together in the corresponding phrase, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used simply to distinguish said components from other said components and do not limit said components in any other aspect (e.g., importance or order). Where any (e.g., 1st) component is referred to as "coupled" or "connected" to another (e.g., 2nd) component, with or without the terms "functionally" or "communicationly," it means that said any component may be connected to said other component directly (e.g., via a wire), wirelessly, or through a third component.

[0205] The term “module” as used in the various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit, for example. A module may be a component formed integrally, or a minimum unit of said component or a part thereof that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0206] Various embodiments of the present document may be implemented as software (e.g., program (140)) comprising one or more instructions stored in a storage medium (e.g., internal memory (136) or external memory (138)) readable by a machine (e.g., electronic device (101)). For example, a processor (e.g., processor (120)) of the machine (e.g., electronic device (101)) may call at least one of the one or more instructions stored in the storage medium and execute it. This enables the machine to be operated to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code that can be executed by an interpreter. The storage medium readable by the machine may be provided in the form of a non-transitory storage medium. Here, 'non-temporary' simply means that the storage medium is a tangible device and does not contain a signal (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily.

[0207] According to one embodiment, the method according to the various embodiments disclosed herein may be provided by being included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or an application store (e.g., Play Store). TM It can be distributed online (e.g., downloaded or uploaded) through ) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created on a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0208] According to various embodiments, each component (e.g., module or program) of the components described above may include a singular or multiple entities, and some of the multiple entities may be separated and placed in other components. According to various embodiments, one or more of the components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Generally or additionally, multiple components (e.g., module or program) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each of the multiple components in the same or similar manner as those performed by the corresponding component among the multiple components prior to integration. According to various embodiments, operations performed by the module, program, or other components may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

[0209] The various embodiments of the present invention disclosed in this specification and drawings are provided merely as specific examples to facilitate the explanation of the technical content of the invention and to aid in understanding the invention, and are not intended to limit the scope of the invention. Accordingly, the scope of the present invention should be interpreted to include all modifications or variations derived based on the technical concept of the invention, in addition to the embodiments disclosed herein.

Claims

1. In a multi-foldable electronic device (101), A housing comprising a first housing (510), a second housing (520), and a third housing (530); At least one sensor (176) disposed in at least one of the first to third housings; A first hinge structure (513) disposed between the first housing and the second housing so that the first housing and the second housing are folded together; A second hinge structure (515) disposed between the second housing and the third housing so that the second housing and the third housing are folded together; A display (160) comprising a first display area (501) positioned corresponding to the first housing, a second display area (503) positioned between the first hinge structure and the second hinge structure corresponding to the second housing, and a third display area (505) positioned corresponding to the third housing; Memory (130) for storing instructions; and The multi-foldable electronic device includes a processor (120), and when the instructions are executed individually and / or collectively by the processor, the multi-foldable electronic device, Detecting the folding state of the first hinge structure based on information obtained from the at least one sensor, and When the above-mentioned first hinge structure is in a folded state, the type of input means is identified, and If the type of the input means is a pen, the window usage size of the third display area is determined based on the type of pen, and If the type of the above input means is a user's finger, the window usage size of the above third display area is determined based on the touch area and shape, and A multi-foldable electronic device that changes the layout of a screen displayed in the third display area based on the window usage size determined above.

2. In paragraph 1, when the instructions are executed individually and / or collectively by the processor, the multi-foldable electronic device, If the above input means is a pen, Identifying the thickness of the pen or the length of the pen nib corresponding to the pen from the above memory, and Calculate the orientation of the above pen and / or the tilt of the above pen, and A multi-foldable electronic device for determining the window usage size of the third display area based on at least one of the thickness of the identified pen, the length of the identified pen tip, the direction of the pen, or the inclination of the pen.

3. In paragraph 2, when the instructions are executed individually and / or collectively by the processor, the multi-foldable electronic device, After performing pairing with the above pen, receive sensing values ​​from the above pen in real time, and A multi-foldable electronic device for calculating the direction of the pen and / or the tilt of the pen based on the received sensing value.

4. In paragraph 1, when the instructions are executed individually and / or collectively by the processor, the multi-foldable electronic device, When the above input means is a user's finger, the touch area and shape of the finger touching the third display area are analyzed, and A multi-foldable electronic device that determines the window usage size of the third display area based on the touch area and shape.

5. In paragraph 4, when the instructions are executed individually and / or collectively by the processor, the multi-foldable electronic device, Identifying the type of finger in real time based on the above touch area and shape, and A multi-foldable electronic device that changes the layout of the screen based on the type of finger identified above.

6. In paragraph 1, when the instructions are executed individually and / or collectively by the processor, the multi-foldable electronic device, When the first hinge structure is in a folded state, the folding angle of the second hinge structure is detected based on information obtained from the at least one sensor, and A multi-foldable electronic device that determines the window usage size of the third display area by further considering the folding angle of the second hinge structure.

7. In paragraph 1, when the instructions are executed individually and / or collectively by the processor, the multi-foldable electronic device, Detecting a touch on the third display area while the layout of the changed screen is displayed in the third display area, A multi-foldable electronic device that determines whether to process the touch based on the area of ​​the touch and the time of the touch.

8. In paragraph 7, when the instructions are executed individually and / or collectively by the processor, the multi-foldable electronic device, Determining whether the area of ​​the above touch and the time of the above touch correspond to a specified usage pattern, A multi-foldable electronic device that processes the touch when the area of ​​the touch and the time of the touch correspond to the specified usage pattern.

9. In paragraph 8, when the instructions are executed individually and / or collectively by the processor, the multi-foldable electronic device, If the area of ​​the touch and the time of the touch correspond to the specified usage pattern, it is determined whether a state change of the multi-foldable electronic device is detected within the specified time, and A multi-foldable electronic device that processes the touch when a state change of the multi-foldable electronic device is detected within the specified time.

10. In paragraph 7, when the instructions are executed individually and / or collectively by the processor, the multi-foldable electronic device, Determining whether the area of ​​the above touch and the time of the above touch correspond to a specified usage pattern, A multi-foldable electronic device that ignores the touch when the area of ​​the touch and the time of the touch do not correspond to the specified usage pattern.

11. A method of operation of a multi-foldable electronic device (101) comprising a housing including a first housing (510), a second housing (520) and a third housing (530), at least one sensor disposed in at least one of the first housing to the third housing, a first hinge structure (513) disposed between the first housing and the second housing so that the first housing and the second housing fold together, a second hinge structure (515) disposed between the second housing and the third housing so that the second housing and the third housing fold together, a first display area (501) disposed corresponding to the first housing, a second display area (503) disposed between the first hinge structure and the second hinge structure corresponding to the second housing, and a third display area (505) disposed corresponding to the third housing. An operation of detecting the folding state of the first hinge structure based on information obtained from at least one sensor; When the above-mentioned first hinge structure is in a folded state, an operation to identify the type of input means; When the type of the input means is a pen, an operation to determine the window usage size of the third display area based on the type of pen; When the type of the above input means is a user's finger, an operation to determine the window usage size of the third display area based on the touch area and shape; and A method including the operation of changing the layout of the screen displayed in the third display area based on the window usage size determined above.

12. In Paragraph 11, the operation determined above is, If the above input means is a pen, An operation to identify the thickness of the pen or the length of the pen nib corresponding to the pen from the memory; An operation to calculate the direction of the above pen and / or the inclination of the above pen; and A method comprising determining the window usage size of the third display area based on at least one of the thickness of the identified pen, the length of the identified pen nib, the direction of the pen, or the inclination of the pen.

13. In Clause 12, the above-mentioned calculating operation is, The operation of receiving a sensing value in real time from the pen after pairing with the pen; and A method comprising the operation of calculating the direction of the pen and / or the tilt of the pen based on the received sensing value.

14. In Paragraph 11, the operation determined above is, If the above input means is the user's finger, The operation of analyzing the touch area and shape of a finger touching the third display area; and An operation to identify the type of finger in real time based on the above touch area and shape; and A method including an action of changing the layout of the screen based on the type of finger identified above.

15. In Paragraph 11, the operation determined above is, An operation of detecting the folding angle of the second hinge structure based on information obtained from the at least one sensor when the first hinge structure is in a folded state; and A method including an operation to determine the window usage size of the third display area by further considering the folding angle of the second hinge structure.

Images